-- Logs begin at Fri 2022-10-28 10:22:52 UTC, end at Fri 2022-10-28 11:25:03 UTC. -- Oct 28 10:52:34.453513 dv-cinder-zed-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 28 10:52:34.462227 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 28 10:52:34 2022] *** Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: nodename: dv-cinder-zed-3-iscsi Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: machine: x86_64 Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: clock source: unix Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: pcre jit disabled Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: detected number of CPU cores: 4 Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: current working directory: / Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: detected binary path: /usr/bin/uwsgi-core Oct 28 10:52:34.464730 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: your processes number limit is 47680 Oct 28 10:52:34.465438 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: your memory page size is 4096 bytes Oct 28 10:52:34.465438 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: detected max file descriptor number: 2048 Oct 28 10:52:34.465438 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: lock engine: pthread robust mutexes Oct 28 10:52:34.465438 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: thunder lock: enabled Oct 28 10:52:34.465438 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 28 10:52:34.465438 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 28 10:52:34.504187 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: Python main interpreter initialized at 0x55750e4889c0 Oct 28 10:52:34.504187 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: python threads support enabled Oct 28 10:52:34.504187 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: your server socket listen backlog is limited to 100 connections Oct 28 10:52:34.504187 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: your mercy for graceful operations on workers is 90 seconds Oct 28 10:52:34.504532 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: mapped 403077 bytes (393 KB) for 2 cores Oct 28 10:52:34.504612 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: *** Operational MODE: preforking *** Oct 28 10:52:34.504853 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: *** uWSGI is running in multiple interpreter mode *** Oct 28 10:52:34.504853 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: spawned uWSGI master process (pid: 105318) Oct 28 10:52:34.505049 dv-cinder-zed-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 28 10:52:34.505747 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: spawned uWSGI worker 1 (pid: 105319, cores: 1) Oct 28 10:52:34.506020 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: spawned uWSGI worker 2 (pid: 105320, cores: 1) Oct 28 10:52:34.506561 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 28 10:52:35.163658 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 10:52:35.165133 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 10:52:36.245361 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 10:52:36.245361 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: from cryptography.utils import int_from_bytes Oct 28 10:52:36.245939 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 10:52:36.245939 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: from cryptography.utils import int_from_bytes Oct 28 10:52:36.246356 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 10:52:36.246356 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: from cryptography.utils import int_from_bytes Oct 28 10:52:36.246785 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 10:52:36.246785 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: from cryptography.utils import int_from_bytes Oct 28 10:52:38.717668 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 10:52:38.718375 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 10:52:38.718706 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 10:52:38.718787 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 10:52:38.718865 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 10:52:38.718932 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 10:52:38.719003 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 10:52:38.719067 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 10:52:38.719130 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 10:52:38.719216 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 10:52:38.719427 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.719520 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.719780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.719877 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.720104 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.720204 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.720424 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.720510 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.720753 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.720857 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.721063 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.721155 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.721382 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.721477 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.721702 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.721794 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.722059 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.722157 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.722325 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.722457 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.722678 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.722841 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.723011 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.723128 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.723331 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.723455 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.723640 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.723779 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.723951 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.724077 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.724268 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.724375 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.724571 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.724685 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.724889 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.725010 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.725193 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.725321 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.725690 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.725820 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.726150 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.726265 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.726460 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.726573 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.726770 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.726883 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.727106 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.727244 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.727420 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.727514 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.727731 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.727822 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.728049 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.728183 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.728357 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.728453 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.728676 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.728765 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.728984 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.729074 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.729379 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.729473 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.729692 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.729783 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.730024 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.730110 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.730341 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.730471 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.730753 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.730856 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.731062 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.731146 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.731372 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.731462 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.731689 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.731780 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.732029 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.732112 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.732341 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.732431 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.732667 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.732751 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.733241 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.733339 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.733559 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.733644 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.733879 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.734035 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.734212 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.734298 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.734519 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.734602 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.734826 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.734921 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.735634 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.735735 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.735951 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.736041 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.736252 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.736334 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.736501 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.736592 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.736806 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.736889 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.737126 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.737227 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.737444 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.737540 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.737765 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.737852 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.738035 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.738124 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.738288 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.738386 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.738605 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.738693 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.738924 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.739017 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.739234 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.739328 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.739540 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.739635 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.739841 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.739940 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.740146 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.740231 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.740451 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.740534 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.740756 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.740848 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.741059 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.741161 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.741364 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.741448 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.741680 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.741771 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.742001 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.742086 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.742328 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.742411 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.742652 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.742787 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.742965 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.743064 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.743359 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.46 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.743459 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.46 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.743610 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.46 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.743721 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.46 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.743916 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.744018 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.744256 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.744345 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.744574 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.744659 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.744880 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.744978 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.745195 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.745279 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.745514 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.745599 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.745820 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.745912 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.746152 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.746256 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.746458 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.746541 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.746776 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.746859 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.747105 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.747197 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.747408 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.747492 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.747656 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.747738 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.747955 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.748048 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.748259 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.748349 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.748577 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.748670 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.748880 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.748984 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.749185 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.749276 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.749488 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.749594 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.749790 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.749969 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.750101 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.750180 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.750410 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.750495 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.750715 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.750837 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.751575 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.751660 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.751880 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.751963 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.752186 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.752271 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.752488 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.752585 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.752799 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.752883 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.753102 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.753196 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.753404 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.753488 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.753721 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.753805 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.754060 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.754148 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.754368 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.754455 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.754671 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.754769 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.754975 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.755076 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.755278 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.755362 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.755595 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.755679 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.755900 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.755984 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.756203 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.756293 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.756522 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.756612 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.756824 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.756915 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.757127 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.757211 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.757440 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.757537 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.757745 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.757830 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.758069 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.758167 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.758386 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.758483 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.758684 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.758767 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.758987 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.759070 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.759302 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.759397 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.759605 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.759689 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.759908 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.760004 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.760208 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.760304 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.760509 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.760593 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.760810 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.760900 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.761112 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.761203 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.761412 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.761495 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.761712 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.761794 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.762029 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.762116 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.762347 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.762428 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.762668 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.762752 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.762974 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.763058 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.763279 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.763369 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.763581 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.763670 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.763896 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.763987 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 10:52:38.764227 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.764310 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.764558 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.764640 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.764894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.764979 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.765209 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.765292 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.765537 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.765642 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.765869 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.765963 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.766209 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.766308 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.766545 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.766647 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.766858 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.766990 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.767183 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.767266 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.767495 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.767586 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.767804 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.767902 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.768115 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.768206 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.768434 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.768538 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.768753 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.768882 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.769083 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.769179 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.769400 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.769492 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.769717 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.769802 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.770053 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.770151 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.770367 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.770460 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.770684 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.770768 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.771009 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.771103 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.771329 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.771422 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.771638 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.771723 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.771978 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.772063 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.772324 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.772409 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.772642 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.772727 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.772968 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.773059 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.773290 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.773373 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.773601 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.773684 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.773911 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.774021 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.774249 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.774348 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.774590 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.774690 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.774903 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.774994 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.775215 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.775304 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.775531 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.775614 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.775844 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.775947 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.776164 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.776254 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.776478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.776570 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.776791 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.776883 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.777104 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.777193 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.777415 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.777497 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.777726 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.777817 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.778086 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.778174 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.778403 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.778503 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.778726 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.778809 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.779041 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.779124 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.779355 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.779437 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.779670 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.779753 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.779983 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.780105 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.780290 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.780373 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.780601 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.780686 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.780906 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.780989 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.781208 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.781296 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.781512 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.781597 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.781821 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.781912 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.782169 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.782271 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.782489 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.782584 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.782822 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.782906 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.783129 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.783225 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.783460 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.783543 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.783766 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.783849 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.784077 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.784173 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.784387 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.784472 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.784695 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.784783 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.784999 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.785092 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.785310 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.785402 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.785618 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.785702 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.785921 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.786036 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.786268 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.786353 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.786580 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.786672 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.786884 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.786969 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.787205 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.787299 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.787511 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.787594 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.787816 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.787900 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.788128 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.788212 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.788439 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.788530 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.788750 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.788835 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.789062 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.789153 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.789371 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.789456 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.789692 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.789776 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.790016 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.790115 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.790329 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.790421 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.790661 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.790745 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.790990 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.791073 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.791296 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.791388 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.791617 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.791706 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.791940 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.792025 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.792256 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.792347 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.792557 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.792640 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.792869 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.792959 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.793168 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.793259 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.793472 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.793563 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.793777 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.793861 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.794109 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.794195 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.794413 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.794552 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.794720 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.794817 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.795026 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.795121 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.795325 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.795415 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.795627 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.795712 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.795946 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.796039 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.796255 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.796350 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.796554 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.796645 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.796855 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.796947 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.797156 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.797248 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.797494 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.797588 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.798331 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.798429 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.798673 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.798756 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.799003 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.799094 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.799322 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.799413 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.799644 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.799734 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.799955 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.800046 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.800271 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.800362 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.800580 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.800663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.800891 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.801053 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.801200 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.801284 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.801509 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.801600 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.801828 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.801920 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.802166 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.802242 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.802482 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.802554 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.802790 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.802872 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.803120 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.803434 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.803756 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.804033 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.804125 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.804351 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.804452 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.804674 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.804787 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.804983 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.805128 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.805302 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.805449 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.805642 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.805777 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.806025 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.806115 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.806319 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.806425 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.806643 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.806727 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.806957 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.807042 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.807274 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.807358 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.807586 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.807670 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.807898 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.807983 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.808211 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.808303 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.808523 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.808619 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.808848 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.808933 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.809159 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.809243 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.809468 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.809550 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.809774 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.809866 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.810113 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.810199 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.810423 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.810518 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.810742 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.810825 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.811047 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.811138 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.811359 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.811453 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.811672 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.811762 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.812000 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.812091 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.812305 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.812396 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.812617 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.812700 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.812934 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.813024 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.813241 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.813371 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.814130 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.814217 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.814445 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.814533 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.814750 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.814836 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.815058 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.815145 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.815368 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.815451 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.815680 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.815765 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.815987 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.816071 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.816296 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.816381 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.816600 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.816686 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.816906 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.816989 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.817213 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.817295 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.817518 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.817604 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.817820 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.817912 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.818158 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.818261 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.818479 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.818564 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.818787 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.818872 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.819099 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.819190 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.819420 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.819503 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.819727 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.819811 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.820030 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.820126 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.820335 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.820418 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.820646 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.820733 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.820948 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.821047 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.821251 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.821335 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.821556 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.821638 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.821899 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.822010 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.822238 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.822332 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.822545 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.822630 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.822860 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.822945 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.823179 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.823269 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.823488 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.823572 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.823802 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.823887 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.824111 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.824195 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.824418 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.824501 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.824731 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.824816 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.825042 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.825132 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.825352 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.825435 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.825664 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.825757 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.825987 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.826072 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.826303 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.826401 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.826614 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.826699 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.826926 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.827008 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.827233 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.827330 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.827545 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.827636 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.827854 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.827938 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.828159 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.828420 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.828506 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.828758 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.828843 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.829102 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.829426 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.829519 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.829777 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.829871 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.830130 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.830217 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.830454 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.830555 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.830785 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.830869 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.831102 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.831195 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.831413 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.831498 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.831727 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.831819 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.832036 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.832131 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.832341 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.832431 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.832653 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.832744 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.832957 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.833039 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.833265 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.833348 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.833569 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.833654 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.833873 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.833956 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.834203 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.834298 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.834509 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.834595 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.834815 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.834944 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.835117 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.835202 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.835421 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.835514 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.835727 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.835813 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.836033 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.836117 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.836338 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.836423 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.836641 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.836732 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.836955 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.837048 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.837261 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.837352 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.837572 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.837656 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.837878 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.837969 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.838209 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.838295 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.838512 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.838596 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.838820 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.838912 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.839166 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.839250 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.839473 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.839558 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.839810 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.839897 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.840118 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.840202 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.840449 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.840532 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.840760 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.840844 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.841068 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.841152 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.841377 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.841462 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.841683 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.841768 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.842015 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.842104 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.842314 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.842398 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.842626 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.842712 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.842935 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.843027 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.843239 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.843336 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.843551 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.843712 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.843859 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.843944 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.844169 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.844265 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.844474 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.844625 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.844799 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.844891 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.845109 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.845205 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.845411 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.845496 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.845729 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.845813 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.846062 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.846167 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.846370 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.846455 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.846675 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.846758 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.846982 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.847066 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.847287 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.847370 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.847587 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.847671 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.847897 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.847982 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.848201 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.848294 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.848511 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.848642 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.848823 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.848908 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.849124 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.849217 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.849427 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.849511 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.849735 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.849820 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.850072 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.850159 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.850377 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.850473 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.850682 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.850765 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.850995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.851078 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.851301 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.851398 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.851600 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.851683 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.851907 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.851990 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.852210 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.852302 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.852513 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.852606 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.852820 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.852911 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.853139 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.853231 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.853449 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.853541 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.853750 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.853832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.854082 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.854169 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.854389 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.854482 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.854694 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.854820 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.854996 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.855116 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.855301 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.855392 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.855599 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.855690 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.855907 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.855992 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.856210 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.856294 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.856512 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.856605 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.856815 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.856907 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.857120 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.857203 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.857429 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.857520 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.857732 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.857824 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.858325 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.858412 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.858629 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.858728 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.858981 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.859066 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.859291 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.859382 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.859612 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.859704 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.859930 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.860021 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.860758 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.860850 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.861076 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.861167 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.861386 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.861470 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.861699 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.861783 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.862032 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.862131 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.862340 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.862423 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.862651 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.862734 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.862965 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.863050 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.863279 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.863375 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.863584 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.863668 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.863891 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.863976 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.864212 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.864296 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.864524 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.864616 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.864835 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.864922 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.865140 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.865223 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.865454 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.865546 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.865773 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.865865 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.866107 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.866198 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.866421 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.866523 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.866732 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.866816 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.867044 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.867136 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.867354 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.867454 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.867663 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.867747 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.867974 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.868059 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.868286 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.868377 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.868594 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.868685 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.868907 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.868993 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.869213 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.869306 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.869519 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.869603 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.869826 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.869919 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.870163 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.870251 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.870470 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.870553 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.870785 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.870870 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.871093 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.871178 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.871397 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.871481 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.871702 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.871792 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.872004 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.872134 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.872315 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.872453 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.872621 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.872769 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.873043 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.873135 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.873355 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.873439 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.873667 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.873759 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.873984 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.874072 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.874301 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.874387 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.874613 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.874714 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.874930 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.875014 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.875243 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.875326 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.875549 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.875641 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.876514 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.876611 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.876829 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.876914 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.877145 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.877248 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.877460 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.877547 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.877764 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.877848 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.878094 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.878184 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.878406 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.878503 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.878713 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.878797 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.879026 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.879118 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.879332 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.879426 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.879639 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.879725 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.879961 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.880055 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.880266 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.880350 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.880587 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.880680 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.881001 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.881087 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.881311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.881395 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.881620 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.881723 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.881991 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.882089 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.882329 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.882424 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.882653 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.882738 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.882963 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.883056 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.883284 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.883368 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.883600 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.883682 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.883905 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.883998 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.884221 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.884306 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.884531 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.884616 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.884853 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.884948 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.885162 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.885249 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.885472 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.885556 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.885789 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.885891 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.886118 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.886206 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.886445 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.886530 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.886773 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.886860 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.887091 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.887176 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.887414 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.887498 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.887790 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.887874 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.888121 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.888206 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.888431 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.888514 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.888739 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.888824 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.889057 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.889145 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.889361 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.889444 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.889664 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.889750 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.890009 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.890110 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.890304 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.890401 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.890610 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.890702 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.890923 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.891006 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.891225 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.891316 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.891554 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.891642 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.891863 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.891956 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.892172 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.892265 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.892477 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.892659 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.892781 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.893000 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.893105 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.893323 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.893409 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.893651 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.893742 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.893994 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.894093 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.894314 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.894399 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.894644 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.894728 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.894952 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.895044 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.895259 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.895351 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.895567 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.895653 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.895870 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.896034 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.896175 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.896260 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.896480 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.896566 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.896821 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.896913 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.897129 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.897213 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.897453 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.897545 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.897777 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.897862 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.898126 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.898225 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.898443 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.898530 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.898785 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.898876 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.899101 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.899186 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.899423 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.899513 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.899736 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.899830 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.900047 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.900131 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.900358 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.900449 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.900664 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.900756 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.900970 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.901069 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.901278 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.901415 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.901584 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.901669 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.901891 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.902017 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.902220 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.902316 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.902528 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.902611 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.902832 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.902925 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.903137 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.903231 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.903443 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.903529 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.903766 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.903853 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.904072 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.904164 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.904379 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.904462 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.904690 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.904782 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.904995 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.905080 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.905298 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.905389 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.905600 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.905722 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.905942 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.906108 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.906270 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.906392 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.906599 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.906724 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.906909 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.907106 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.907257 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.907397 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.907575 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.907721 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.907892 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.908043 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.908210 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.908366 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.908525 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.908689 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.908847 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.909012 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.909161 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.909325 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.909476 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.909647 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.909784 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.909968 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.910134 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.910314 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.910452 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.910770 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.910855 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.911164 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.911287 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.911486 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.911616 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.911805 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.911931 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.912125 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.912245 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.912445 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.912560 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.912762 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.912874 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.913084 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.913170 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.913400 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.913493 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.913716 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.913809 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.914058 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.914146 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.914391 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.914488 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.914717 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.914809 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.915038 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.915123 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.915364 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.915453 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.915685 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.915768 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.916002 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.916093 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.916318 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.916405 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.916635 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.916719 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.916949 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.917041 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.917299 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.917384 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.917611 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.917705 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.917914 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.918030 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.918255 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.918349 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.918570 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.918654 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.918885 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.918970 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.919193 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.919275 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.919503 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.919596 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.919816 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.919902 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.920127 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.920217 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.920434 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.920517 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.920742 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.920827 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.921054 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.921139 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.921366 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.921458 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.921688 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.921783 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.922033 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.922120 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.922353 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.922449 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.923179 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.923272 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.923618 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.923757 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.923863 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.924027 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.924328 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.924415 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.924645 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.924730 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.924977 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.925072 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.925302 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.925393 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.925622 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.925718 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.925937 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.926040 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.926283 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.926376 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.926593 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.926683 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.926904 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.926989 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.927205 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.927289 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.927517 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.927601 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.927824 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.927909 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.928127 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.928217 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.928449 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.928534 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.928754 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.928839 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.929057 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.929147 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.929375 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.929472 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.929683 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.929907 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.930243 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.930343 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.930593 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.930685 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.930910 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.931001 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.931240 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.931333 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.931559 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.931687 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.931872 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.931957 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.932180 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.932263 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.932498 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.932582 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.932804 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.932888 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.933114 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.933207 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.933433 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.933518 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.933741 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.933832 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.934066 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.934153 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.934375 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.934462 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.934688 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.934772 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.934995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.935086 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.935299 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.935385 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.935604 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.935687 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.935904 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.935995 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.936230 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.936321 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.936534 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.936618 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.936838 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.936928 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.937149 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.937234 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.937458 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.937541 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.937759 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.937850 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.938086 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.938173 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.938935 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.939030 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.939243 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.939337 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.939553 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.939636 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.939854 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.939936 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.940162 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.940255 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.940466 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.940558 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.940780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.940865 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.941122 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.941214 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.941432 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.941516 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.941736 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.941819 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.942073 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.942160 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.942378 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.942472 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.942682 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.942775 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.942995 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.943080 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.943302 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.943394 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.943626 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.943711 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.943941 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.944026 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.944250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.944338 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.944596 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.944688 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.944911 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.945134 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.945445 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.945537 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.945780 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.945877 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.946126 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.946225 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.946460 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.946549 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.946773 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.946866 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.947085 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.947177 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.947393 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.947485 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.947702 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.947795 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.948022 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.948105 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.948333 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.948423 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.948646 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.948730 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.948960 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.949044 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.949274 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.949365 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.949582 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.949666 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.949893 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.950004 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.950229 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.950312 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.950539 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.950625 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.950850 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.950942 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.951161 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.951260 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.951469 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.951553 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.951778 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.951908 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.952088 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.952182 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.952397 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.952481 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.952712 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.952805 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.953050 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.953147 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.953368 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.953459 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.953686 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.953770 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.954046 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.954136 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.954365 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.954449 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.954673 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.954767 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.954984 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.955068 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.955290 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.955374 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.955601 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.955685 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.955917 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.956004 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.956234 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.956331 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.956542 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.956644 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.956846 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.957215 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.957499 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.957584 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.957814 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.957907 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.958161 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.958260 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.958486 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.958580 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.958799 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.958897 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.959129 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.959214 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.959451 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.959535 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.959761 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.959846 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.960082 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.960167 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.960395 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.960481 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.960724 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.960810 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.961062 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.961146 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.961388 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.961472 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.961699 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.961784 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.962027 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.962115 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.962336 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.962437 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.962642 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.962727 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.962951 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.963035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.963270 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.963362 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.963579 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.963670 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.963883 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.963968 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.964188 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.964280 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.964497 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.964588 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.964798 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.964882 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.965103 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.965203 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.965406 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.965489 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.965711 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.965806 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.966037 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.966129 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.966343 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.966434 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.966647 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.966735 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.966951 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.967036 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.967251 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.967345 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.967552 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.967638 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.967899 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.967991 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.968161 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.968244 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.968461 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.968544 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.968765 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.968851 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.969071 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.969161 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.969373 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.969466 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.969699 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.969790 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.970051 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.970152 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.970386 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.970479 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.970698 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.970782 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.971016 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.971099 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.971326 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.971419 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.971641 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.971724 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.971949 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.972034 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.972270 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.972366 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.972588 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.972671 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.972900 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.972985 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.973212 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.973304 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.973539 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.973623 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.973846 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.973931 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.974185 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.974271 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.974497 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.974581 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.974807 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.974891 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.975114 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.975204 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.975422 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.975508 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.975747 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.975840 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.976055 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.976146 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.976361 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.976489 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.976685 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.976779 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.976999 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.977090 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.977315 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.977400 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.977627 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.977723 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.977937 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.978091 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.978301 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.978385 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.978612 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.978699 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.978929 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.979020 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.979238 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.979329 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.979549 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.979634 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.979856 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.979939 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.980171 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.980257 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.980482 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.980574 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.980792 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.980882 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.981108 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.981205 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.981425 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.981523 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.981749 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.981834 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.982083 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.982182 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.982398 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.982485 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.982721 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.982810 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.983299 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.983399 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.983629 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.983713 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.983945 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.984028 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.984288 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.984381 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.984603 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.984695 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.984913 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.984996 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.985739 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.985830 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.986016 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 10:52:38.986143 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 10:52:38.986308 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105319) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 10:52:39.138825 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO dbcounter [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Registered counter for database nova_api Oct 28 10:52:39.139056 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO dbcounter [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Registered counter for database nova_api Oct 28 10:52:39.160898 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_db.sqlalchemy.engines [None req-292104f3-0222-4ad1-9221-25456a5634be None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105319) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.162643 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_db.sqlalchemy.engines [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105320) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.167791 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writer thread running {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.169139 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writer thread running {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.223872 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.context [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),38276e5b-4682-4472-aad8-debabc8dd457(cell1) {{(pid=105319) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 10:52:39.226316 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.context [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),38276e5b-4682-4472-aad8-debabc8dd457(cell1) {{(pid=105320) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 10:52:39.227850 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_concurrency.lockutils [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105319) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:52:39.228323 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_concurrency.lockutils [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105319) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:52:39.229875 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_concurrency.lockutils [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105319) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:52:39.230356 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:52:39.230610 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_concurrency.lockutils [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105319) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:52:39.230844 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:52:39.231049 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_concurrency.lockutils [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105319) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:52:39.232408 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:52:39.233258 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_concurrency.lockutils [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105319) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:52:39.233364 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:52:39.233818 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:52:39.235817 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO dbcounter [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Registered counter for database nova_cell0 Oct 28 10:52:39.236386 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:52:39.238474 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO dbcounter [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Registered counter for database nova_cell0 Oct 28 10:52:39.239808 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO dbcounter [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Registered counter for database nova_cell1 Oct 28 10:52:39.242569 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO dbcounter [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Registered counter for database nova_cell1 Oct 28 10:52:39.246944 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_db.sqlalchemy.engines [None req-292104f3-0222-4ad1-9221-25456a5634be None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105319) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.248136 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_db.sqlalchemy.engines [None req-292104f3-0222-4ad1-9221-25456a5634be None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105319) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.251065 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_db.sqlalchemy.engines [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105320) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.254505 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_db.sqlalchemy.engines [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105320) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.254824 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writer thread running {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.256998 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writer thread running {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.257610 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writer thread running {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.396464 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writer thread running {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.405569 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO dbcounter [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Registered counter for database nova_cell0 Oct 28 10:52:39.406438 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO dbcounter [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Registered counter for database nova_cell0 Oct 28 10:52:39.413187 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG oslo_db.sqlalchemy.engines [None req-292104f3-0222-4ad1-9221-25456a5634be None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105319) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.416680 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_db.sqlalchemy.engines [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105320) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 10:52:39.417959 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writer thread running {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.421472 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writer thread running {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 10:52:39.754443 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: WARNING keystonemiddleware.auth_token [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 10:52:39.757928 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: WARNING keystonemiddleware.auth_token [None req-292104f3-0222-4ad1-9221-25456a5634be None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 10:52:39.761630 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: WARNING keystonemiddleware.auth_token [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 10:52:39.765315 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: WARNING keystonemiddleware.auth_token [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 10:52:40.103084 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: WARNING keystonemiddleware.auth_token [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 10:52:40.109297 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: WARNING keystonemiddleware.auth_token [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 10:52:40.111955 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: WARNING keystonemiddleware.auth_token [None req-292104f3-0222-4ad1-9221-25456a5634be None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 10:52:40.114590 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55750e4889c0 pid: 105320 (default app) Oct 28 10:52:40.116685 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:52:40.117876 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] 10.222.0.46 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001758 Oct 28 10:52:40.117966 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: WARNING keystonemiddleware.auth_token [None req-292104f3-0222-4ad1-9221-25456a5634be None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 10:52:40.118325 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 788 bytes} [Fri Oct 28 10:52:40 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:52:40.123616 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55750e4889c0 pid: 105319 (default app) Oct 28 10:54:00.872306 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:00.873488 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001618 Oct 28 10:54:00.874105 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 2/2] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:00.886426 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_api:SELECT=3 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:00.887667 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_cell1:SELECT=3 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:00.888763 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_cell0:SELECT=3 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:00.890119 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:01.766540 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-e295181b-6896-4447-bcca-2b9f8777cda0 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:01.798658 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-e295181b-6896-4447-bcca-2b9f8777cda0 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.917807 Oct 28 10:54:01.799492 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 1/3] 10.222.0.46 () {60 vars in 1292 bytes} [Fri Oct 28 10:54:00 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 920 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:04.427829 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:04.428903 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-008d04ca-5b9d-4606-a9ba-f8217771b61a None None] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001626 Oct 28 10:54:04.429375 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 3/4] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:04 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:04.871342 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1113df25-65d1-4636-97ac-06908a199220 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "id": "c1", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "ram": 256, "disk": 1, "name": "cirros256"}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:04.924130 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1113df25-65d1-4636-97ac-06908a199220 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.488627 Oct 28 10:54:04.924722 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 2/5] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:04 2022] POST /compute/v2.1/flavors => generated 407 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:04.940033 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_api:SELECT=3 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:04.942980 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_cell0:SELECT=3 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:04.943651 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_cell1:SELECT=3 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:04.944338 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:05.153475 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-c0643c3b-512c-4455-8777-bb040c0656fc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:05.222787 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-c0643c3b-512c-4455-8777-bb040c0656fc admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.288701 Oct 28 10:54:05.223163 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 4/6] 10.222.0.46 () {64 vars in 1336 bytes} [Fri Oct 28 10:54:04 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:07.824210 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1113df25-65d1-4636-97ac-06908a199220 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:07.825248 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1113df25-65d1-4636-97ac-06908a199220 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Oct 28 10:54:07.825917 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 3/7] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:07 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:08.471567 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-68958969-ad41-4c57-a3da-11ea7a0083e2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "name": "ds512M", "rxtx_factor": 1.0, "disk": 5, "swap": 0, "id": "d1", "vcpus": 1}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:08.512447 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-68958969-ad41-4c57-a3da-11ea7a0083e2 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.679680 Oct 28 10:54:08.513233 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 5/8] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:07 2022] POST /compute/v2.1/flavors => generated 404 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:08.525521 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-532e715c-e113-4e5b-b95b-87fbdd92a1a3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:08.565084 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-532e715c-e113-4e5b-b95b-87fbdd92a1a3 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.043191 Oct 28 10:54:08.565681 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 4/9] 10.222.0.46 () {64 vars in 1336 bytes} [Fri Oct 28 10:54:08 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:11.149933 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-68958969-ad41-4c57-a3da-11ea7a0083e2 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:11.150984 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-68958969-ad41-4c57-a3da-11ea7a0083e2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001490 Oct 28 10:54:11.151407 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 6/10] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:11.572672 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-75b37eb0-1fbc-43ee-ba30-d0aa61b35214 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "disk": 10, "vcpus": 1, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 1024, "id": "d2", "swap": 0, "name": "ds1G"}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:11.592071 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-75b37eb0-1fbc-43ee-ba30-d0aa61b35214 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.433176 Oct 28 10:54:11.592724 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 5/11] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:11 2022] POST /compute/v2.1/flavors => generated 404 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:11.605227 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-fac927f2-b584-49e9-9af7-67bf435c3aab admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:11.632832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-fac927f2-b584-49e9-9af7-67bf435c3aab admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031293 Oct 28 10:54:11.633425 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 7/12] 10.222.0.46 () {64 vars in 1336 bytes} [Fri Oct 28 10:54:11 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:14.279391 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-75b37eb0-1fbc-43ee-ba30-d0aa61b35214 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:14.281000 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-75b37eb0-1fbc-43ee-ba30-d0aa61b35214 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002042 Oct 28 10:54:14.281403 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 6/13] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:14 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:14.691021 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-73672f6a-8153-47d5-bd9a-1ea0d66f1f45 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "swap": 0, "id": "d3", "rxtx_factor": 1.0, "name": "ds2G", "disk": 10, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:14.718068 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-73672f6a-8153-47d5-bd9a-1ea0d66f1f45 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.422529 Oct 28 10:54:14.719694 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 8/14] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:14 2022] POST /compute/v2.1/flavors => generated 404 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:14.731813 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-c34b33e5-ebdf-4e18-8ee5-337b7dee18b5 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:14.758463 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-c34b33e5-ebdf-4e18-8ee5-337b7dee18b5 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030801 Oct 28 10:54:14.759035 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 7/15] 10.222.0.46 () {64 vars in 1336 bytes} [Fri Oct 28 10:54:14 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:17.438586 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-73672f6a-8153-47d5-bd9a-1ea0d66f1f45 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:17.439536 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-73672f6a-8153-47d5-bd9a-1ea0d66f1f45 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001519 Oct 28 10:54:17.440003 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 9/16] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:17.872756 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-beb4765c-123b-44a6-be4c-9e90475ee4e3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "disk": 20, "name": "ds4G", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 4096, "os-flavor-access:is_public": true, "id": "d4", "vcpus": 4}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:17.891872 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-beb4765c-123b-44a6-be4c-9e90475ee4e3 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.445605 Oct 28 10:54:17.892439 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 8/17] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:17 2022] POST /compute/v2.1/flavors => generated 404 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:17.905642 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-96a8d0c0-833e-4581-b38f-2caa25f4c0a8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:17.933795 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-96a8d0c0-833e-4581-b38f-2caa25f4c0a8 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032058 Oct 28 10:54:17.934952 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 10/18] 10.222.0.46 () {64 vars in 1336 bytes} [Fri Oct 28 10:54:17 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:20.531308 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-beb4765c-123b-44a6-be4c-9e90475ee4e3 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:20.532254 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-beb4765c-123b-44a6-be4c-9e90475ee4e3 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001477 Oct 28 10:54:20.532661 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 9/19] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:20 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:20.964327 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-949f9e9d-4843-4cba-b42c-f60db94dee34 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:20.989124 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-949f9e9d-4843-4cba-b42c-f60db94dee34 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.450078 Oct 28 10:54:20.989656 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 11/20] 10.222.0.46 () {60 vars in 1292 bytes} [Fri Oct 28 10:54:20 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:54:23.724826 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-beb4765c-123b-44a6-be4c-9e90475ee4e3 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:23.725858 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-beb4765c-123b-44a6-be4c-9e90475ee4e3 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001519 Oct 28 10:54:23.726330 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 10/21] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:24.117247 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-c9322e59-b36d-45f6-afc3-46f4c21df149 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "1", "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "vcpus": 1, "os-flavor-access:is_public": true, "name": "m1.tiny", "ram": 512, "rxtx_factor": 1.0}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:24.141551 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-c9322e59-b36d-45f6-afc3-46f4c21df149 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.409079 Oct 28 10:54:24.142236 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 12/22] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:23 2022] POST /compute/v2.1/flavors => generated 402 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:24.154139 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-38855044-0c57-4351-8605-89117b84d931 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:24.180698 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-38855044-0c57-4351-8605-89117b84d931 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030155 Oct 28 10:54:24.181274 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 11/23] 10.222.0.46 () {64 vars in 1333 bytes} [Fri Oct 28 10:54:24 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:26.738836 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-c9322e59-b36d-45f6-afc3-46f4c21df149 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:26.739855 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-c9322e59-b36d-45f6-afc3-46f4c21df149 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001495 Oct 28 10:54:26.740260 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 13/24] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:27.114352 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-392b0005-aa5e-41db-bf53-ffd08a8b129a admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "name": "m1.small", "disk": 20, "id": "2", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 1, "swap": 0, "rxtx_factor": 1.0}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:27.132904 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-392b0005-aa5e-41db-bf53-ffd08a8b129a admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.386499 Oct 28 10:54:27.133806 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 12/25] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:26 2022] POST /compute/v2.1/flavors => generated 405 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:27.146643 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-c87f7524-3083-4566-8379-b0cf71b4c7b9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:27.169796 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-c87f7524-3083-4566-8379-b0cf71b4c7b9 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027686 Oct 28 10:54:27.170965 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 14/26] 10.222.0.46 () {64 vars in 1333 bytes} [Fri Oct 28 10:54:27 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:29.766840 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-392b0005-aa5e-41db-bf53-ffd08a8b129a admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:29.767860 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-392b0005-aa5e-41db-bf53-ffd08a8b129a admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001507 Oct 28 10:54:29.768481 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 13/27] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:29 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:30.201220 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-225199fc-164e-4656-86c2-58defb30ec25 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "name": "m1.medium", "ram": 4096, "rxtx_factor": 1.0, "swap": 0, "id": "3", "vcpus": 2, "disk": 40, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:30.223246 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-225199fc-164e-4656-86c2-58defb30ec25 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.448712 Oct 28 10:54:30.223968 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 15/28] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:29 2022] POST /compute/v2.1/flavors => generated 406 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:30.236113 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-deefa932-5066-48b3-b7bf-9e673c32cbc4 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:30.264959 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-deefa932-5066-48b3-b7bf-9e673c32cbc4 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032758 Oct 28 10:54:30.265553 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 14/29] 10.222.0.46 () {64 vars in 1333 bytes} [Fri Oct 28 10:54:30 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:32.888110 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-225199fc-164e-4656-86c2-58defb30ec25 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:32.889698 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-225199fc-164e-4656-86c2-58defb30ec25 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002038 Oct 28 10:54:32.890396 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 16/30] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:32 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:33.326687 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-ff9cee64-a292-4432-a699-ebef87d8ee73 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "rxtx_factor": 1.0, "name": "m1.large", "disk": 80, "ram": 8192, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "4", "os-flavor-access:is_public": true, "swap": 0}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:33.332480 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:33.349204 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-ff9cee64-a292-4432-a699-ebef87d8ee73 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.451220 Oct 28 10:54:33.362708 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-39eb2734-1866-4fe4-9b61-5879395666bb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:33.367198 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 15/31] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:32 2022] POST /compute/v2.1/flavors => generated 405 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:33.386701 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-39eb2734-1866-4fe4-9b61-5879395666bb admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027912 Oct 28 10:54:33.387284 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 17/32] 10.222.0.46 () {64 vars in 1333 bytes} [Fri Oct 28 10:54:33 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:35.984169 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-ff9cee64-a292-4432-a699-ebef87d8ee73 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:35.985443 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-ff9cee64-a292-4432-a699-ebef87d8ee73 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001725 Oct 28 10:54:35.985858 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 16/33] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:36.440962 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-1bfd434c-55b7-4576-b422-631673998824 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "id": "5", "vcpus": 8, "rxtx_factor": 1.0, "swap": 0, "ram": 16384, "name": "m1.xlarge"}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:36.447489 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:36.463745 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-1bfd434c-55b7-4576-b422-631673998824 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.471615 Oct 28 10:54:36.464313 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 18/34] 10.222.0.46 () {64 vars in 1283 bytes} [Fri Oct 28 10:54:35 2022] POST /compute/v2.1/flavors => generated 408 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:54:36.477947 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-6b6344be-c55f-4bc5-af54-8d39546df12a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:54:36.512407 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-6b6344be-c55f-4bc5-af54-8d39546df12a admin admin] 10.222.0.46 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038036 Oct 28 10:54:36.512960 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 17/35] 10.222.0.46 () {64 vars in 1333 bytes} [Fri Oct 28 10:54:36 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:54:53.327079 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-1bfd434c-55b7-4576-b422-631673998824 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:53.328137 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-1bfd434c-55b7-4576-b422-631673998824 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001505 Oct 28 10:54:53.328585 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 19/36] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:54:53 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:53.338994 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:54:53.712730 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-60e8feb3-85c8-4cd8-8755-eb6d55fd62f9 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:53.731587 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-60e8feb3-85c8-4cd8-8755-eb6d55fd62f9 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.397012 Oct 28 10:54:53.732124 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 18/37] 10.222.0.46 () {60 vars in 1292 bytes} [Fri Oct 28 10:54:53 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:54:55.027339 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-1bfd434c-55b7-4576-b422-631673998824 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:55.028525 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-1bfd434c-55b7-4576-b422-631673998824 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Oct 28 10:54:55.028937 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 20/38] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:54:55 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:55.540292 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-356cf1f2-c324-41de-801f-4b8ccb9e3393 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:55.546105 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.wsgi [None req-356cf1f2-c324-41de-801f-4b8ccb9e3393 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-356cf1f2-c324-41de-801f-4b8ccb9e3393 admin admin] Request method failure captured: Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Accept: application/json Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Accept-Encoding: gzip, deflate Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Connection: keep-alive Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Content-Length: 0 Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Host: 10.222.0.46 Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Openstack-Api-Version: compute 2.92 Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Openstack-System-Scope: None Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: User-Agent: python-novaclient Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Auth-Token: gAAAAABjW7T-aToHgYYDgUksMpx02qDFb-CRXobKfUuvK9iEOSEy3tZA-AKI79ewUpn4ZdU7GsFllhLgLwR1vEd5VsvX6YEpui_kkUTzu7S4lldmowjfag7Hvv6zySWOOfILLdCAJ0ptfOyNqkMWZi2Djn_zLei0BIB2RNNkA0X6_B10UeptEmg Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Domain-Id: None Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Domain-Name: None Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Identity-Status: Confirmed Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Is-Admin-Project: True Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Domain-Id: default Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Domain-Name: Default Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Name: admin Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Role: reader,member,admin Oct 28 10:54:55.549306 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Roles: reader,member,admin Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2b351b41bbb64be2822e6354d987b945"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant: admin Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant-Name: admin Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User: admin Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Domain-Id: default Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Domain-Name: Default Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Id: 79c132c0cbf74b0899faa3036130f049 Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Name: admin Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: method: > Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: exception: Flavor 42 could not be found. Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: action_args: {'id': '42'} Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 10:54:55.550923 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:55.551853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.552995 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-356cf1f2-c324-41de-801f-4b8ccb9e3393 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105319) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 10:54:55.554086 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-356cf1f2-c324-41de-801f-4b8ccb9e3393 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.386229 Oct 28 10:54:55.554086 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 19/39] 10.222.0.46 () {60 vars in 1190 bytes} [Fri Oct 28 10:54:55 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 387 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 10:54:55.560854 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-73298d6d-c42f-486b-bc90-a18da2b96413 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:55.583158 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-73298d6d-c42f-486b-bc90-a18da2b96413 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026303 Oct 28 10:54:55.583714 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 21/40] 10.222.0.46 () {60 vars in 1210 bytes} [Fri Oct 28 10:54:55 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:54:55.594550 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-7de64e9f-5361-4497-b1ae-8fe321789df4 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:55.616945 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-7de64e9f-5361-4497-b1ae-8fe321789df4 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027304 Oct 28 10:54:55.617470 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 20/41] 10.222.0.46 () {60 vars in 1210 bytes} [Fri Oct 28 10:54:55 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:54:55.626823 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-997f8b87-b25d-4719-88ed-4f1ad272680d admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:55.631857 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.wsgi [None req-997f8b87-b25d-4719-88ed-4f1ad272680d admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-997f8b87-b25d-4719-88ed-4f1ad272680d admin admin] Request method failure captured: Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Accept: application/json Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Accept-Encoding: gzip, deflate Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Connection: keep-alive Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Content-Length: 0 Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Host: 10.222.0.46 Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Openstack-Api-Version: compute 2.92 Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Openstack-System-Scope: None Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: User-Agent: python-novaclient Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Auth-Token: gAAAAABjW7T-aToHgYYDgUksMpx02qDFb-CRXobKfUuvK9iEOSEy3tZA-AKI79ewUpn4ZdU7GsFllhLgLwR1vEd5VsvX6YEpui_kkUTzu7S4lldmowjfag7Hvv6zySWOOfILLdCAJ0ptfOyNqkMWZi2Djn_zLei0BIB2RNNkA0X6_B10UeptEmg Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Domain-Id: None Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Domain-Name: None Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Identity-Status: Confirmed Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Is-Admin-Project: True Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Domain-Id: default Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Domain-Name: Default Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Name: admin Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Role: reader,member,admin Oct 28 10:54:55.635097 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Roles: reader,member,admin Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2b351b41bbb64be2822e6354d987b945"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant: admin Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant-Name: admin Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User: admin Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Domain-Id: default Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Domain-Name: Default Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Id: 79c132c0cbf74b0899faa3036130f049 Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Name: admin Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: method: > Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: exception: Flavor 42 could not be found. Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: action_args: {'id': '42'} Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 10:54:55.636478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:55.637241 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:55.639038 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-997f8b87-b25d-4719-88ed-4f1ad272680d admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105320) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 10:54:55.640021 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-997f8b87-b25d-4719-88ed-4f1ad272680d admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.013419 Oct 28 10:54:55.640021 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 22/42] 10.222.0.46 () {60 vars in 1190 bytes} [Fri Oct 28 10:54:55 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 10:54:56.850509 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-7de64e9f-5361-4497-b1ae-8fe321789df4 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:56.851538 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-7de64e9f-5361-4497-b1ae-8fe321789df4 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001519 Oct 28 10:54:56.851971 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 21/43] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:54:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:57.370044 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-3bd563e5-813f-4962-94ce-041e7a1e2803 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:57.375140 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.wsgi [None req-3bd563e5-813f-4962-94ce-041e7a1e2803 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-3bd563e5-813f-4962-94ce-041e7a1e2803 admin admin] Request method failure captured: Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Accept: application/json Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Accept-Encoding: gzip, deflate Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Connection: keep-alive Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Content-Length: 0 Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Host: 10.222.0.46 Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Openstack-Api-Version: compute 2.92 Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Openstack-System-Scope: None Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: User-Agent: python-novaclient Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Auth-Token: gAAAAABjW7UA6E7CxQe9qM_NbBS3JGjzV9VBvTgB2cEmBDLh1s7Q3leJnAUXMwZSkgXF9OQRtlhLLQaBccw2AF-80C-vX2HUrLwT1xVzSLELFw9qZWQBhVI19E3OAhN5YDnLvdNpzBlplhHloPJyIS_w7VRw_WZeh-SVp6TS3F8zPpqIa07r4ig Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Domain-Id: None Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Domain-Name: None Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Identity-Status: Confirmed Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Is-Admin-Project: True Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Domain-Id: default Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Domain-Name: Default Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Name: admin Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Role: reader,member,admin Oct 28 10:54:57.376663 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Roles: reader,member,admin Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2b351b41bbb64be2822e6354d987b945"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant: admin Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant-Name: admin Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User: admin Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Domain-Id: default Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Domain-Name: Default Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Id: 79c132c0cbf74b0899faa3036130f049 Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Name: admin Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: method: > Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: exception: Flavor 84 could not be found. Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: action_args: {'id': '84'} Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 10:54:57.378143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:57.379035 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.380478 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-3bd563e5-813f-4962-94ce-041e7a1e2803 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105320) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 10:54:57.381897 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-3bd563e5-813f-4962-94ce-041e7a1e2803 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.389324 Oct 28 10:54:57.381897 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 23/44] 10.222.0.46 () {60 vars in 1190 bytes} [Fri Oct 28 10:54:56 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 390 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 10:54:57.388478 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-3303b7f2-e215-45a5-af4c-3a6a19faef4f admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:57.403853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-3303b7f2-e215-45a5-af4c-3a6a19faef4f admin admin] 10.222.0.46 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018971 Oct 28 10:54:57.404390 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 22/45] 10.222.0.46 () {60 vars in 1210 bytes} [Fri Oct 28 10:54:57 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:54:57.413500 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-48c32433-4ab4-441b-b11e-8730fba7eab2 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:57.428515 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-48c32433-4ab4-441b-b11e-8730fba7eab2 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018548 Oct 28 10:54:57.429028 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 24/46] 10.222.0.46 () {60 vars in 1210 bytes} [Fri Oct 28 10:54:57 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 10:54:57.437951 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-d8f1a89c-35b4-483f-9352-d8fa9a727210 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:57.442813 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.wsgi [None req-d8f1a89c-35b4-483f-9352-d8fa9a727210 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-d8f1a89c-35b4-483f-9352-d8fa9a727210 admin admin] Request method failure captured: Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Accept: application/json Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Accept-Encoding: gzip, deflate Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Connection: keep-alive Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Content-Length: 0 Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Host: 10.222.0.46 Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Openstack-Api-Version: compute 2.92 Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Openstack-System-Scope: None Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: User-Agent: python-novaclient Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Auth-Token: gAAAAABjW7UA6E7CxQe9qM_NbBS3JGjzV9VBvTgB2cEmBDLh1s7Q3leJnAUXMwZSkgXF9OQRtlhLLQaBccw2AF-80C-vX2HUrLwT1xVzSLELFw9qZWQBhVI19E3OAhN5YDnLvdNpzBlplhHloPJyIS_w7VRw_WZeh-SVp6TS3F8zPpqIa07r4ig Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Domain-Id: None Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Domain-Name: None Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Identity-Status: Confirmed Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Is-Admin-Project: True Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Domain-Id: default Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Domain-Name: Default Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Name: admin Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Role: reader,member,admin Oct 28 10:54:57.444311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Roles: reader,member,admin Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2b351b41bbb64be2822e6354d987b945"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant: admin Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant-Name: admin Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User: admin Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Domain-Id: default Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Domain-Name: Default Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Id: 79c132c0cbf74b0899faa3036130f049 Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Name: admin Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: method: > Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: exception: Flavor 84 could not be found. Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: action_args: {'id': '84'} Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 10:54:57.445894 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:57.446901 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:57.448293 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-d8f1a89c-35b4-483f-9352-d8fa9a727210 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105319) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 10:54:57.449607 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-d8f1a89c-35b4-483f-9352-d8fa9a727210 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011137 Oct 28 10:54:57.449607 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 23/47] 10.222.0.46 () {60 vars in 1190 bytes} [Fri Oct 28 10:54:57 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 28 10:54:58.656569 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-48c32433-4ab4-441b-b11e-8730fba7eab2 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:58.657608 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-48c32433-4ab4-441b-b11e-8730fba7eab2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001468 Oct 28 10:54:58.658044 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 25/48] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:54:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:54:59.175151 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-9ee529b1-a1d0-42f0-a9a2-430511a4411c admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:59.180775 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.wsgi [None req-9ee529b1-a1d0-42f0-a9a2-430511a4411c admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-9ee529b1-a1d0-42f0-a9a2-430511a4411c admin admin] Request method failure captured: Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Accept: application/json Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Accept-Encoding: gzip, deflate Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Connection: keep-alive Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Content-Length: 0 Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Host: 10.222.0.46 Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Openstack-Api-Version: compute 2.92 Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Openstack-System-Scope: None Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: User-Agent: python-novaclient Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Auth-Token: gAAAAABjW7UCIgtnm6VMM2b7VQDt7JPYJ2W2OQT7xpKWF9M_44ESXvrbMvzfCwfSR3qxpD7l29OrWF7066HI-F9aVnAWsRpZs0jd4yEfIQOzGJ8dWZWr4kLw2pliT9YKfMjOQ9fo0f9mak-tXvOXI8wS7y_ug9jZmnQ-v4WGka-5AmDmD1zMLZg Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Domain-Id: None Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Domain-Name: None Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Identity-Status: Confirmed Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Is-Admin-Project: True Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Domain-Id: default Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Domain-Name: Default Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Project-Name: admin Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Role: reader,member,admin Oct 28 10:54:59.182311 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Roles: reader,member,admin Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2b351b41bbb64be2822e6354d987b945"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant: admin Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-Tenant-Name: admin Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User: admin Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Domain-Id: default Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Domain-Name: Default Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Id: 79c132c0cbf74b0899faa3036130f049 Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: X-User-Name: admin Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: method: > Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: exception: Flavor 451 could not be found. Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: action_args: {'id': '451'} Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 10:54:59.183793 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:59.184780 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.186250 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-9ee529b1-a1d0-42f0-a9a2-430511a4411c admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105319) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 10:54:59.187512 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-9ee529b1-a1d0-42f0-a9a2-430511a4411c admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.390169 Oct 28 10:54:59.187512 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 24/49] 10.222.0.46 () {60 vars in 1193 bytes} [Fri Oct 28 10:54:58 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 391 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 28 10:54:59.193675 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-1efd94b4-7a52-41b4-bed7-35280f4dfb6d admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:59.209019 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-1efd94b4-7a52-41b4-bed7-35280f4dfb6d admin admin] 10.222.0.46 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018894 Oct 28 10:54:59.209549 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 26/50] 10.222.0.46 () {60 vars in 1210 bytes} [Fri Oct 28 10:54:59 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 10:54:59.219399 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:59.234622 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019012 Oct 28 10:54:59.235160 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 25/51] 10.222.0.46 () {60 vars in 1210 bytes} [Fri Oct 28 10:54:59 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:54:59.244277 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-946dd5c0-02c1-48cc-b0bf-1c10a0adb281 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:54:59.249274 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.wsgi [None req-946dd5c0-02c1-48cc-b0bf-1c10a0adb281 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-946dd5c0-02c1-48cc-b0bf-1c10a0adb281 admin admin] Request method failure captured: Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Accept: application/json Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Accept-Encoding: gzip, deflate Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Connection: keep-alive Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Content-Length: 0 Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Host: 10.222.0.46 Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Openstack-Api-Version: compute 2.92 Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Openstack-System-Scope: None Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: User-Agent: python-novaclient Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Auth-Token: gAAAAABjW7UCIgtnm6VMM2b7VQDt7JPYJ2W2OQT7xpKWF9M_44ESXvrbMvzfCwfSR3qxpD7l29OrWF7066HI-F9aVnAWsRpZs0jd4yEfIQOzGJ8dWZWr4kLw2pliT9YKfMjOQ9fo0f9mak-tXvOXI8wS7y_ug9jZmnQ-v4WGka-5AmDmD1zMLZg Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Domain-Id: None Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Domain-Name: None Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Identity-Status: Confirmed Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Is-Admin-Project: True Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Openstack-Nova-Api-Version: 2.92 Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Domain-Id: default Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Domain-Name: Default Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Project-Name: admin Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Role: reader,member,admin Oct 28 10:54:59.250832 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Roles: reader,member,admin Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2b351b41bbb64be2822e6354d987b945"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2b351b41bbb64be2822e6354d987b945"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant: admin Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant-Id: 2b351b41bbb64be2822e6354d987b945 Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-Tenant-Name: admin Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User: admin Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Domain-Id: default Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Domain-Name: Default Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Id: 79c132c0cbf74b0899faa3036130f049 Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: X-User-Name: admin Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: method: > Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: exception: Flavor 451 could not be found. Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: action_args: {'id': '451'} Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 10:54:59.252143 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:59.253119 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: ERROR nova.api.openstack.wsgi Oct 28 10:54:59.254403 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-946dd5c0-02c1-48cc-b0bf-1c10a0adb281 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105320) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 10:54:59.255694 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-946dd5c0-02c1-48cc-b0bf-1c10a0adb281 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011477 Oct 28 10:54:59.255694 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 27/52] 10.222.0.46 () {60 vars in 1193 bytes} [Fri Oct 28 10:54:59 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 28 10:55:00.454369 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:00.455436 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001547 Oct 28 10:55:00.455836 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 26/53] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:55:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:55:01.095727 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-72e43a50-409c-4f6b-bea4-1f9ac3e61fd5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:55:01.115248 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-72e43a50-409c-4f6b-bea4-1f9ac3e61fd5 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.523286 Oct 28 10:55:01.116804 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 28/54] 10.222.0.46 () {64 vars in 1235 bytes} [Fri Oct 28 10:55:00 2022] POST /compute/v2.1/flavors => generated 404 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 10:55:02.327537 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:02.328561 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001439 Oct 28 10:55:02.328965 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 27/55] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:55:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:55:02.845396 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-6303fa2f-e1d8-47ba-8e42-750e2acb2a1e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:55:02.864214 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-6303fa2f-e1d8-47ba-8e42-750e2acb2a1e admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.399524 Oct 28 10:55:02.864764 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 29/56] 10.222.0.46 () {64 vars in 1235 bytes} [Fri Oct 28 10:55:02 2022] POST /compute/v2.1/flavors => generated 405 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:55:04.081998 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:04.083021 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001450 Oct 28 10:55:04.083417 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 28/57] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:55:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:55:04.608179 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-d2415931-4d22-4ec4-8e65-f83662aea5e7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:55:04.626765 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-d2415931-4d22-4ec4-8e65-f83662aea5e7 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.400469 Oct 28 10:55:04.627306 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 30/58] 10.222.0.46 () {64 vars in 1235 bytes} [Fri Oct 28 10:55:04 2022] POST /compute/v2.1/flavors => generated 407 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 10:55:05.840711 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:05.841752 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 Oct 28 10:55:05.842195 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 29/59] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:55:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:55:06.363529 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-2e28d39f-e02c-4d61-8443-8c246aa9d0d9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 10:55:06.381892 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-2e28d39f-e02c-4d61-8443-8c246aa9d0d9 admin admin] 10.222.0.46 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.401866 Oct 28 10:55:06.382539 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 31/60] 10.222.0.46 () {64 vars in 1235 bytes} [Fri Oct 28 10:55:05 2022] POST /compute/v2.1/flavors => generated 409 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:55:07.599760 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:07.600772 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Oct 28 10:55:07.601164 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 30/61] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:55:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 10:55:08.114527 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-785c20a6-a765-4c16-bc5b-70772371bbc9 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:08.135796 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-785c20a6-a765-4c16-bc5b-70772371bbc9 admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.397732 Oct 28 10:55:08.136378 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 32/62] 10.222.0.46 () {60 vars in 1202 bytes} [Fri Oct 28 10:55:07 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:55:58.076253 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:58.077301 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-1e2b556d-08f8-4688-a1b0-520a41610ae2 admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001484 Oct 28 10:55:58.077701 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 31/63] 10.222.0.46 () {54 vars in 931 bytes} [Fri Oct 28 10:55:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:55:58.088371 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:55:58.467486 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-310a7045-0046-4ce1-924e-e32727b8fe9e admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:58.487962 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-310a7045-0046-4ce1-924e-e32727b8fe9e admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.404017 Oct 28 10:55:58.488507 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 33/64] 10.222.0.46 () {60 vars in 1292 bytes} [Fri Oct 28 10:55:58 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 10:55:58.512571 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-39a1a60d-df42-4fe5-982a-d4cb68d9f05c admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:58.516022 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_api:SELECT=14 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:55:58.521278 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-39a1a60d-df42-4fe5-982a-d4cb68d9f05c admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012915 Oct 28 10:55:58.521819 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 32/65] 10.222.0.46 () {60 vars in 1283 bytes} [Fri Oct 28 10:55:58 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:55:58.533372 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-2e975233-a2e0-4052-ac9e-dc15e8ce890f admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:58.540579 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-2e975233-a2e0-4052-ac9e-dc15e8ce890f admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010843 Oct 28 10:55:58.541091 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 34/66] 10.222.0.46 () {60 vars in 1286 bytes} [Fri Oct 28 10:55:58 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:55:58.552422 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:58.559987 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011314 Oct 28 10:55:58.560517 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 33/67] 10.222.0.46 () {60 vars in 1286 bytes} [Fri Oct 28 10:55:58 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:55:58.571561 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-c7d47eae-af3b-4f8e-ba20-93433535dc1a admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:55:58.579288 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-c7d47eae-af3b-4f8e-ba20-93433535dc1a admin admin] 10.222.0.46 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011167 Oct 28 10:55:58.579810 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 35/68] 10.222.0.46 () {60 vars in 1283 bytes} [Fri Oct 28 10:55:58 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 10:59:20.779853 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:20.780871 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Oct 28 10:59:20.781269 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 34/69] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:59:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:59:20.928548 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_api:SELECT=6 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:59:21.085016 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:21.089755 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:21.090274 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:21.090711 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:21.091429 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:21.091882 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:21.092305 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:21.128535 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-62457786-0c84-42f5-94a1-8c9caff5e4ea admin admin] 10.222.0.46 "GET /compute/v2.1/os-services" status: 200 len: 782 microversion: 2.92 time: 0.201702 Oct 28 10:59:21.129104 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 36/70] 10.222.0.46 () {60 vars in 1193 bytes} [Fri Oct 28 10:59:20 2022] GET /compute/v2.1/os-services => generated 782 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:59:22.202607 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:22.203093 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Oct 28 10:59:22.203692 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 35/71] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:59:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:59:22.390844 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:22.394255 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:22.394738 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:22.395171 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:22.395966 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:22.396419 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:22.396842 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:22.416261 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-23f859bb-c793-41e8-982b-e64cd8a750ba admin admin] 10.222.0.46 "GET /compute/v2.1/os-services" status: 200 len: 782 microversion: 2.92 time: 0.077554 Oct 28 10:59:22.416753 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 37/72] 10.222.0.46 () {60 vars in 1193 bytes} [Fri Oct 28 10:59:22 2022] GET /compute/v2.1/os-services => generated 782 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 10:59:33.886678 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:33.887693 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001469 Oct 28 10:59:33.888089 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 36/73] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:59:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:59:34.031242 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_cell0:SELECT=4 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:59:34.031954 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_cell1:SELECT=4 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:59:34.032548 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG dbcounter [-] [105320] Writing DB stats nova_api:SELECT=8 {{(pid=105320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:59:34.079947 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:34.083343 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:34.084409 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:34.084841 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:34.085556 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:34.086029 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:34.086449 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:34.111061 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-f5eb3c73-ef7f-4d06-a072-c87b708bc116 admin admin] 10.222.0.46 "GET /compute/v2.1/os-services" status: 200 len: 1026 microversion: 2.92 time: 0.084192 Oct 28 10:59:34.111551 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 38/74] 10.222.0.46 () {60 vars in 1193 bytes} [Fri Oct 28 10:59:34 2022] GET /compute/v2.1/os-services => generated 1026 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 10:59:35.176253 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG nova.api.openstack.wsgi [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] Calling method '>' {{(pid=105319) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:35.177890 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: INFO nova.api.openstack.requestlog [None req-39c7d612-b15e-40f6-a64f-f0f51419b6de admin admin] 10.222.0.46 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001422 Oct 28 10:59:35.178336 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: [pid: 105319|app: 0|req: 37/75] 10.222.0.46 () {56 vars in 1074 bytes} [Fri Oct 28 10:59:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 10:59:35.364540 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG nova.api.openstack.wsgi [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] Calling method '>' {{(pid=105320) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 10:59:35.368075 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:35.368575 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:35.369002 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:35.369714 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 10:59:35.370216 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 10:59:35.370633 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: DEBUG oslo_concurrency.lockutils [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105320) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 10:59:35.391704 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: INFO nova.api.openstack.requestlog [None req-f69660f7-df1c-45c5-9531-872eab8986c3 admin admin] 10.222.0.46 "GET /compute/v2.1/os-services" status: 200 len: 1026 microversion: 2.92 time: 0.077788 Oct 28 10:59:35.392195 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: [pid: 105320|app: 0|req: 39/76] 10.222.0.46 () {60 vars in 1193 bytes} [Fri Oct 28 10:59:35 2022] GET /compute/v2.1/os-services => generated 1026 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:00:30.632674 dv-cinder-zed-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 28 11:00:30.633231 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: Fri Oct 28 11:00:30 2022 - graceful shutdown triggered... Oct 28 11:00:30.633705 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: Gracefully killing worker 1 (pid: 105319)... Oct 28 11:00:30.634099 dv-cinder-zed-3-iscsi devstack@n-api.service[105320]: Gracefully killing worker 2 (pid: 105320)... Oct 28 11:00:30.760160 dv-cinder-zed-3-iscsi devstack@n-api.service[105319]: DEBUG dbcounter [-] [105319] Writing DB stats nova_api:SELECT=4 {{(pid=105319) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:00:31.634379 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: worker 1 buried after 1 seconds Oct 28 11:00:31.634822 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: worker 2 buried after 1 seconds Oct 28 11:00:31.634822 dv-cinder-zed-3-iscsi devstack@n-api.service[105318]: goodbye to uWSGI. Oct 28 11:00:31.635973 dv-cinder-zed-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 28 11:00:31.636534 dv-cinder-zed-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 28 11:00:31.639370 dv-cinder-zed-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 28 11:00:31.648858 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 28 11:00:31 2022] *** Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: nodename: dv-cinder-zed-3-iscsi Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: machine: x86_64 Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: clock source: unix Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: pcre jit disabled Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: detected number of CPU cores: 4 Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: current working directory: / Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: detected binary path: /usr/bin/uwsgi-core Oct 28 11:00:31.650962 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: your processes number limit is 47680 Oct 28 11:00:31.651669 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: your memory page size is 4096 bytes Oct 28 11:00:31.651669 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: detected max file descriptor number: 2048 Oct 28 11:00:31.651669 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: lock engine: pthread robust mutexes Oct 28 11:00:31.651669 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: thunder lock: enabled Oct 28 11:00:31.651669 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 28 11:00:31.651669 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 28 11:00:31.691662 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: Python main interpreter initialized at 0x55c2ca6cb9c0 Oct 28 11:00:31.691662 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: python threads support enabled Oct 28 11:00:31.691662 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: your server socket listen backlog is limited to 100 connections Oct 28 11:00:31.691662 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: your mercy for graceful operations on workers is 90 seconds Oct 28 11:00:31.692126 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: mapped 403077 bytes (393 KB) for 2 cores Oct 28 11:00:31.692126 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: *** Operational MODE: preforking *** Oct 28 11:00:31.692403 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: *** uWSGI is running in multiple interpreter mode *** Oct 28 11:00:31.692403 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: spawned uWSGI master process (pid: 120522) Oct 28 11:00:31.692627 dv-cinder-zed-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 28 11:00:31.695087 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: spawned uWSGI worker 1 (pid: 120543, cores: 1) Oct 28 11:00:31.695087 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: spawned uWSGI worker 2 (pid: 120544, cores: 1) Oct 28 11:00:31.695087 dv-cinder-zed-3-iscsi devstack@n-api.service[120522]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 28 11:00:32.362141 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 11:00:32.371360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 28 11:00:33.523522 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 11:00:33.523522 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: from cryptography.utils import int_from_bytes Oct 28 11:00:33.524280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 11:00:33.524280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: from cryptography.utils import int_from_bytes Oct 28 11:00:33.534678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 11:00:33.534678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: from cryptography.utils import int_from_bytes Oct 28 11:00:33.535238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 11:00:33.535238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: from cryptography.utils import int_from_bytes Oct 28 11:00:35.954822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 11:00:35.955384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 11:00:35.955615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 11:00:35.955986 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 11:00:35.956252 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 11:00:35.956651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.956978 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.957505 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.957832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.958194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.958671 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.958851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.959703 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.960042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.960308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.960562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 11:00:35.960932 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 11:00:35.961039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.961122 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.961255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 11:00:35.961562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.961686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 11:00:35.961767 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.961909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 11:00:35.962127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.962312 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.962471 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.962643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.962752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.962968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.963070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.963292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.963386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.963627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.963865 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.963983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965440 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965670 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965831 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.965921 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966082 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966301 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-zed-3-iscsi {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966606 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966846 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.966955 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.967117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.967261 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.967474 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.967690 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.967946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.968153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.968398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.968614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.968614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.969012 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.969300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.969584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.969839 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.970133 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.970325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.970545 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.970707 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.970948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.970948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.971327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.971514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.971781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.971932 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.972083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.972256 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.972516 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.972666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.972818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.973003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.973162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.973307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-zed-3-iscsi {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.973449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.973605 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.973748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.973897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.974096 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.974261 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.974437 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.974437 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.975367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.975639 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.975879 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.976034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.976188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.976338 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.976482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.976633 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.976823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.976973 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.977125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.977312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.977487 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.977681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.977840 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.978129 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.978371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.978574 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.978756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.978908 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.979151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.979347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.979536 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.979690 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.979842 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.979985 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.980142 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.980286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.980439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.980581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.980730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.980730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.981018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.981162 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.981303 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.981463 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.981605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.981797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.981951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.982129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.982302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.982463 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.982609 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.982815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.983304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.983486 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.983643 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.983807 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.983968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.46 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.984155 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.984337 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.46 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.984493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.984635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.984785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.984936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.985127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.985273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.985470 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.985663 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.985888 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.985888 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.986268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.986512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.986671 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.986898 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.987058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.987211 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.987362 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.987539 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.987687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.987859 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.988018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.988189 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.988333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.988573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.46 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.988756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.988948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.46 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.989126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.989285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.989429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.989579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.989737 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.989880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.990075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.990267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.990456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.990640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.990858 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.991079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.991270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.991432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.991583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.991758 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.991921 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.992072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.992230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.992374 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.992526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.992675 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.992816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.992989 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.993142 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.993340 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.993576 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.993770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.993945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.994134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.994326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.994470 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.994611 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.994770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.994770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.995048 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.995205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.995375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.995532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.995730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.995913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.996112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.996289 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.996443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.996596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.996756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.996947 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.997113 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.997257 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.997409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.997589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.997747 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.997890 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.998153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.998301 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.998495 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.998680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.998865 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.999042 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.999212 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.999358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.999507 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.999670 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:35.999848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.000003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.000003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.000283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.000438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.000589 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.000739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.000880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.001035 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.001196 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.001379 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.001523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.001712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.001910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.002144 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.002300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.002794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.003007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.003164 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.003396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.003549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.003712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.003856 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.004005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.004178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.004332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.004475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.004714 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.004922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.005071 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.005268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.005496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.005682 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.005836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.006097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.006322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.006505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.006650 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.006873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.007094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.007277 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.007453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.007614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.007766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.007938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.007938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.008228 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.008479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.008677 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.008836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.009024 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.009181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.009324 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.009468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.009619 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.009824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.010035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.010255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.010405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.010592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.010781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.010989 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 11:00:36.010989 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.011344 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.011495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.011654 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.011798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.011940 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.012124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.012277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.012428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.012581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.012735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.012887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.013074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.013275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.013436 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.013645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.013823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.014060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.014230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.014436 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.014624 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.014813 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.015003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.015164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.015309 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.015478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.015668 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.015838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.016038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.016193 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.016337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.016532 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.016724 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.016948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.017418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.017634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.017811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.017969 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.018152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.018374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.018586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.018836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.019019 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.019200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.019351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.019508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.019661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.019811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.019969 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.020113 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.020264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.020415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.020558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.020699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.020854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.021004 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.021156 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.021299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.021440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.022218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.022405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.022561 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.022833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.023008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.023008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.023321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.023321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.023598 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.023757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.023900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.024042 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.024191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.024341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.024499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.024661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.024806 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.024972 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.025120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.025120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.025438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.025644 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.025835 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.026026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.026182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.026325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.026532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.026719 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.026936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.027111 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.027305 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.027458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.027616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.027768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.027949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.028098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.028248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.028400 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.028544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.028745 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.028912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.029064 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.029246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.029246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.029536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.029756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.029907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.030088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.030233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.030416 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.030573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.030757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.030999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.031166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.031367 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.031552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.031709 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.031852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.032001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.032145 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.032293 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.032502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.032658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.032802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.033046 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.033258 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.033525 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.033721 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.034635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.034824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.034971 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.035162 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.035325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.035478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.035622 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.035765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.035916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.036066 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.036228 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.036391 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.036571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.036723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.036867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.037074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.037897 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.038084 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.038314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.038521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.038668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.038857 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.039011 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.039166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.039336 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.039496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.039656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.039820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.039964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.040113 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.040256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.040408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.040559 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.040711 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.040894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.041040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.041255 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.041485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.041682 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.041834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.042037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.042229 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.042376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.042530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.042741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.042938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.043247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.043401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.043582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.043745 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.043900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.044054 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.044206 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.044358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.044510 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.044655 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.044812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.044963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.045139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.045294 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.045445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.045608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.045754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.045904 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.046088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.046238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.046412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.046573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.046573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.046909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.047134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.047334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.047489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.047647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.047804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.047997 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.048230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.048417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.048570 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.048764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.048980 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.049132 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.049343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.049494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.049647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.049873 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.050110 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.050300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.050445 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.050676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.050917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.051119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.051271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.051450 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.051608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.051752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.051907 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.052058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.052246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.052388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.052575 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.052810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.053029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.053225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.053442 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.053593 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.053736 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.053952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.054179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.054360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.054514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.054778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.054937 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.055185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.055347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.055491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.055641 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.055784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.055927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.055927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.056227 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.056227 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.056508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.056658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.056801 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.056963 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.057140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.057292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.057481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.057673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.057890 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.058130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.058369 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.058573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.058776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.058952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.059135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.059135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.059438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.059438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.059719 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.059719 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.060034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.060034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.060364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.060609 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.060824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.060977 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.061163 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.061320 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.061465 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.061626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.061769 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.061912 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.062122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.062279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.062425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.062567 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.062710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.062710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.063028 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.063028 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.063380 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.063556 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.063690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.063863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.063996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.064156 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.064294 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.064453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.064576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.064695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.064865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.065033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.065187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.065348 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.065480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.065645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.065769 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.065884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.066075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.066218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.066367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.066624 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.066779 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.066951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.067094 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.067233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.067375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.067514 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.067646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.067785 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.067916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.068053 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.068232 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.068372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.068524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.068653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.068784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.068922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.069067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.069199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.069335 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.069473 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.069611 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.069741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.069882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070725 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070864 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.070996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.071126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.071255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.071404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.071537 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.071666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.071807 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.071939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.072069 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.072213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.072345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.072482 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.072613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.072832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.072968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.073112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.073243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.073381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.073512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.073641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.073771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.073771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.074050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.074190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.074339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.074339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.074607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.074738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.074877 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075139 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075410 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075549 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.075939 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.076086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.076227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.076358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.076496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.076496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.076749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.076894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.077026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.077165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.077296 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.077433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.077572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.077717 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.077856 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.078040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.078231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.078378 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.078515 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.078644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.078780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.078918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079061 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.079988 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.080126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.080264 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.080403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.080531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.080667 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.080803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.080932 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.081074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.081211 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.081356 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.081493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.081630 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.081759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.081896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.082129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.082274 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.082408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.082538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.082676 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.082816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.082948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.083077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.083215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.083345 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.083484 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.083623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.083762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.083893 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.084532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.084720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.084863 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.084995 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.085126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.085256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.085387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.085526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.085671 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.085803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.085936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.086131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.086273 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.086405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.086598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.086598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.086857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.086996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.087135 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.087266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.087405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.087536 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.087675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.087806 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.087937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.088080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.088211 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.088380 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.088511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.088511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.088765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.088909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.089049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.089188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.089320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.089457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.089589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.089733 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.089873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090399 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090742 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.090872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091410 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091539 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.091957 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.092095 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.092234 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.092365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.092495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.092632 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.092843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.092977 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.093114 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.093245 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.093383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.093528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.093667 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.093806 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.093940 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094107 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094336 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094515 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094711 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.094999 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095084 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095174 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095288 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095577 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095745 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.095918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096513 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.096960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097144 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097144 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.097927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098034 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098297 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098574 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098659 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.098923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099005 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099096 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099379 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.099643 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100382 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100577 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.100903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101357 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.101958 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102059 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102131 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102214 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102214 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.102955 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.103177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.103282 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.103488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.103576 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.103798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.103910 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.104111 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.104204 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.104423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.104516 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.104737 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.104829 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.105050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.105141 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.105368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.105500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.105678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.105777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.106009 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.106104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.106322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.106415 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.106637 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.106729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.106950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.107049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.107267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.107419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.107588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.107688 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.108153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.108263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.108469 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.108574 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.108786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.108878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.109101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.109198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.109413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.109510 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.109724 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.109824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.110072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.110180 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.110394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.110496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.110714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.110807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.111032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.111120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.111348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.111448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.111665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.111766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.111980 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.112082 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.112300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.112422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.112619 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.112731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.112930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.113043 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.113263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.113373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.113576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.113683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.113950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.114158 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.114322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.114475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.114619 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.114771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.114925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.115078 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.115221 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.115399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.115553 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.115722 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.115936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.116096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.116252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.116410 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.116557 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.116699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.116849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.117007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.117151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.117301 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.117453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.117596 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.117739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.117908 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.118122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.118271 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.118434 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.118618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.118776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.118920 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.119064 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.119215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.119357 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.119499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.119760 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.119913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.120065 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.120222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.120365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.120515 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.120668 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.120817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.120968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.121110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.121282 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.121428 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.121593 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.121745 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.121908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.122076 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.122244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.122484 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.122642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.122794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.122939 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.123089 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.123257 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.123421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.123582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.123749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.123892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.124035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.124196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.124341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.124490 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.124639 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.124782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.124937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.125080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.125221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.125389 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.125848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.125848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.126286 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.126286 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.126584 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.126736 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.126887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.127029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.127184 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.127335 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.127485 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.127636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.127786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.127937 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.128089 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.128241 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.128384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.128544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.128738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.128903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.129054 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.129205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.129348 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.129501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.129652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.129896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.130066 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.130066 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.130353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.130503 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.130669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.130821 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.131002 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.131155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.131306 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.131306 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.131597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.131743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.131904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.132099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.132253 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.132397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.132588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.132743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.132887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.133036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.133181 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.133330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.133482 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.133626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.133771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.133913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.134076 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.134230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.134389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.134543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.134686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.134838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.134981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.135123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.135274 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.135423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.135566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.135708 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.135851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.136002 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.136162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.136380 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.136541 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.136692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.136844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.136998 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.137148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.137291 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.137433 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.137575 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.137717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.137872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.138049 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.138199 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.138352 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.138512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.138670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.138865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.139017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.139159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.139302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.139443 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.139595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.139746 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.139907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.140056 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.140214 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.140368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.140524 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.140668 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.140810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.140952 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.141102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.141245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.141405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.141548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.141690 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.141839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.142008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.142172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.142334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.142486 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.142628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.142771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.142922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.143074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.143217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.143366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.143508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.143664 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.143807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.143962 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.144106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.144266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.144417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.144560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.144710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.144862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.145012 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.145155 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.145306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.145449 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.145642 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.145788 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.145930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.146096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.146251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.146495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.147239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.147386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.147529 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.147672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.147815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.147957 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.148116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.148267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.148420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.148563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.148723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.148915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.149079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.149223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.149381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.149525 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.149675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.149827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.149994 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.150153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.150306 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.150448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.150598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.150764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.150909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.151058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.151210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.151388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.151548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.151731 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.151883 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.152032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.152182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.152387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.152540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.152699 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.152853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.153035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.153184 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.153358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.153591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.153746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.153892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.154097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.154248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.154399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.154556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.154700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.154861 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.155005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.155156 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.155306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.155458 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.155601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.155752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.155918 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.156069 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.156244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.156396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.156625 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.156786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.156953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.157099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.157251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.157397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.157538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.157692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.157844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.158031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.158184 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.158338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.158481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.158634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.158777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.158973 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.159125 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.159268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.159421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.159565 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.159706 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.159870 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.160023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.160166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.160309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.160462 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.160691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.160837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.160990 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.161148 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.161306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.161457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.161625 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.161784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.161941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.162749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.162923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.163089 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.163254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.163413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.163572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.163731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.163882 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.164045 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.164196 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.164398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.164398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.164702 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.164853 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.165027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.165179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.165329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.165486 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.165637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.165786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.165949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.166124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.166286 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.166445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.166613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.166847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.167021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.167185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.167336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.167493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.167643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.167800 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.167957 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.168107 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.168256 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.168405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.168553 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.168715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.168873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.169068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.169244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.169398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.169554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.169717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.169868 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.170075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.170233 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.170233 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.170536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.170687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.170837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.170995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.171145 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.171313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.171472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.171653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.171812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.171962 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.172119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.172277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.172435 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.172586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.172733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.172890 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.173059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.173223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.173384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.173542 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.173694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.173842 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.174010 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.174179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.174340 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.174497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.174656 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.174814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.174971 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.175131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.175298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.175456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.175614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.175772 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.175923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.176081 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.176245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.176395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.176551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.176708 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.176891 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.177131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.177289 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.177457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.177607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.177756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.177944 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.178146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.178298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.178447 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.178603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.178753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.178912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.179063 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.179263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.179422 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.179586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.179742 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.179903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.180053 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.180209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.180359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.180516 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.180673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.180836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.180836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.181122 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.181271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.181429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.181429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.181736 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.181893 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.182076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.182237 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.182388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.182550 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.182701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.182857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.183007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.183157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.183307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.183480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.183631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.183787 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.183944 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.184094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.184250 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.184409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.184567 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.184716 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.184872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.184872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.185171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.185329 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.185489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.185638 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.185786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.185934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.186110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.186229 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.186324 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.186465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.186579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.186780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.186886 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.187092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.187183 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.187427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.187533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.187746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.187857 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.188066 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.188161 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.188381 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.188481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.188694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.188788 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.189010 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.189102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.189330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.189422 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.189646 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.189745 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.189962 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.190077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.190303 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.190397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.190620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.190715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.190939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.191032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.191254 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.191360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.191578 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.191684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.191893 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.191986 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.192207 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.192310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.192526 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.192627 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.192843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.192944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.193166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.193290 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.193585 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.193766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.193922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.194109 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.194279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.194439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.194599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.194751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.194908 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.195060 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.195219 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.195418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.195579 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.195739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.195896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.196049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.196209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.196361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.196510 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.196659 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.196659 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.196953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.197116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.197362 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.197787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.197950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.198158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.198322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.198473 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.198637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.198637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.198925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.199074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.199238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.199439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.199596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.199757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.199916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.200075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.200226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.200374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.200531 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.200704 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.200869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.200869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.201223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.201386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.201568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.201750 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.201911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.201911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202378 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202470 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202728 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202870 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.202947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.203032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.203251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.203362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.203571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.203737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.203905 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.204016 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.204252 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.204660 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.204822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.204985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.205138 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.205298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.205468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.205627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.205785 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.205936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.206153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.206308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.206308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.206593 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.206742 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.206899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.206899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.207202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.207379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.207613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.207613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.207904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.208063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.208223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.208375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.208524 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.208681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.208831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.208994 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.209752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.209752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.210096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.210096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.210404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.210569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.210721 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.210878 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.211027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.211185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.211346 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.211497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.211654 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.211812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.211971 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.212132 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.212291 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.212450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.212600 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.212757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.212907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.213058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.213216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.213377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.213557 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.213730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.213887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.214097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.214267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.214696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.214854 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.215006 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.215155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.215320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.215472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.215630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.215789 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.215941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.216116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.216279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.216441 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.216593 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.216750 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.216911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.217068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.217228 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.217379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.217528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.217774 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.217933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.218135 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.218296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.218447 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.218614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.218766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.218915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.219073 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.219224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.219373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.219530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.219723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.219881 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.220042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.220193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.220350 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.220512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.220662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.220894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.221055 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.221213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.221372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.221523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.221680 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.221841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.222012 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.222180 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.222332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.222500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.222651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.222810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.222961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.223118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.223282 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.223445 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.223607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.223766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.223927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.224078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.224228 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.224385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.224558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.225307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.225476 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.225636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.225795 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.225952 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.226149 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.226304 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.226453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.226602 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.226751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.226908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.227083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 11:00:36.227243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.227243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.227243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.227243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 11:00:36.227243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120543) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 11:00:36.287350 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO dbcounter [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Registered counter for database nova_api Oct 28 11:00:36.288577 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO dbcounter [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Registered counter for database nova_api Oct 28 11:00:36.307881 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_db.sqlalchemy.engines [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120544) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.309293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_db.sqlalchemy.engines [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120543) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.314595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writer thread running {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.315781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writer thread running {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.373888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.context [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),38276e5b-4682-4472-aad8-debabc8dd457(cell1) {{(pid=120544) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 11:00:36.375023 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.context [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),38276e5b-4682-4472-aad8-debabc8dd457(cell1) {{(pid=120543) load_cells /opt/stack/nova/nova/context.py:464}} Oct 28 11:00:36.377822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:00:36.378327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:00:36.378942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:00:36.379418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:00:36.379931 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:00:36.380676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:00:36.381562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:00:36.381681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:00:36.382323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:00:36.382774 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:00:36.383694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:00:36.384761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:00:36.385672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO dbcounter [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Registered counter for database nova_cell0 Oct 28 11:00:36.386783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO dbcounter [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Registered counter for database nova_cell0 Oct 28 11:00:36.389863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO dbcounter [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Registered counter for database nova_cell1 Oct 28 11:00:36.390940 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO dbcounter [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Registered counter for database nova_cell1 Oct 28 11:00:36.398254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_db.sqlalchemy.engines [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120544) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.401016 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_db.sqlalchemy.engines [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120543) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.401842 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_db.sqlalchemy.engines [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120543) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.404290 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_db.sqlalchemy.engines [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120544) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.408310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writer thread running {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.408627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writer thread running {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.411118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writer thread running {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.414561 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writer thread running {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.560070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO dbcounter [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Registered counter for database nova_cell0 Oct 28 11:00:36.562485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO dbcounter [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Registered counter for database nova_cell0 Oct 28 11:00:36.567255 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_db.sqlalchemy.engines [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120544) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.571008 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_db.sqlalchemy.engines [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120543) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 11:00:36.572528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writer thread running {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.576206 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writer thread running {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 11:00:36.896538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: WARNING keystonemiddleware.auth_token [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 11:00:36.896931 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: WARNING keystonemiddleware.auth_token [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 11:00:36.903950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: WARNING keystonemiddleware.auth_token [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 11:00:36.904160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: WARNING keystonemiddleware.auth_token [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 11:00:37.252475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: WARNING keystonemiddleware.auth_token [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 11:00:37.258480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: WARNING keystonemiddleware.auth_token [None req-c3f1a8de-74c6-4b30-9eb3-36ae0125de4a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 11:00:37.263244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: WARNING keystonemiddleware.auth_token [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 28 11:00:37.263771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c2ca6cb9c0 pid: 120544 (default app) Oct 28 11:00:37.269191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: WARNING keystonemiddleware.auth_token [None req-6acda777-82ff-411b-9e4c-c0f8c9146c1c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 28 11:00:37.274529 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c2ca6cb9c0 pid: 120543 (default app) Oct 28 11:01:14.872493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:14.873570 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:14.874722 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:14.876038 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:16.597421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b9785725-b309-4ba1-8748-e9f564042829 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Action: 'create', calling method: .version_select at 0x7f42fb5ea3a0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1703205245"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:18.193524 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b9785725-b309-4ba1-8748-e9f564042829 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 3.332108 Oct 28 11:01:18.194197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 1/1] 10.222.0.46 () {60 vars in 1155 bytes} [Fri Oct 28 11:01:14 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 3334 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 28 11:01:18.214602 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:18.215502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:18.218167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:18.221179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:18.283762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7bea515a-9a4d-4429-bf81-8e10b2bfaed0 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42fb4cc280>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:18.843696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-90e23310-f450-4008-922f-19d7d61d785e tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb5e9040>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:18.898144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7bea515a-9a4d-4429-bf81-8e10b2bfaed0 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/bf6942d0-7a97-49f8-9882-423567a75cf9" status: 200 len: 351 microversion: 2.1 time: 0.632272 Oct 28 11:01:18.898720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 2/2] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:18 2022] GET /compute/v2.1/os-volumes/bf6942d0-7a97-49f8-9882-423567a75cf9 => generated 351 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:01:19.877897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-90e23310-f450-4008-922f-19d7d61d785e tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 200 len: 346 microversion: 2.1 time: 1.669930 Oct 28 11:01:19.878762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 1/3] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:18 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 346 bytes in 1672 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:19.893053 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-eaee5884-de61-4cb3-b35a-22889f73d9d8 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Action: 'create', calling method: .version_select at 0x7f42fb4cc940>, body: {"snapshot": {"volume_id": "ae826d88-35f4-42d4-9353-c1b3f5e95bb9", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1878632925"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:20.269402 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-eaee5884-de61-4cb3-b35a-22889f73d9d8 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.380769 Oct 28 11:01:20.270029 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 3/4] 10.222.0.46 () {60 vars in 1162 bytes} [Fri Oct 28 11:01:19 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:20.300002 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0759b322-dbed-41b9-9e09-aea3ecfd1adc tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb4af1f0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:20.399741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0759b322-dbed-41b9-9e09-aea3ecfd1adc tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.120867 Oct 28 11:01:20.400701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 2/5] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:20 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:20.413195 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ec27dbe6-0e1b-430f-8395-dc96450b8b3c tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb455af0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:20.475788 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ec27dbe6-0e1b-430f-8395-dc96450b8b3c tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.066763 Oct 28 11:01:20.476395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 4/6] 10.222.0.46 () {58 vars in 1140 bytes} [Fri Oct 28 11:01:20 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:20.491521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5fab23f1-7904-4132-9a08-61f859825ea9 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb4af940>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:20.542405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5fab23f1-7904-4132-9a08-61f859825ea9 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.056515 Oct 28 11:01:20.543168 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 3/7] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:20 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:21.556205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4261ee13-9e17-4888-9a6f-2c6c30bc1e59 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb4571f0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:21.615326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4261ee13-9e17-4888-9a6f-2c6c30bc1e59 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.063549 Oct 28 11:01:21.616480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 5/8] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:21 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:22.630512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ef700b8e-8840-4e0e-8862-aa87e66a5b28 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb4afb80>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:22.694364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ef700b8e-8840-4e0e-8862-aa87e66a5b28 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.069199 Oct 28 11:01:22.695242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 4/9] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:22 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:23.377461 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-333019278", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "networks": [{"uuid": "9ae7ccc8-0f91-48dd-b29a-d699bdc8437f"}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:23.587962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] validate_networks() for [('9ae7ccc8-0f91-48dd-b29a-d699bdc8437f', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:01:23.709940 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-599c6ea5-009c-4949-9bb9-ae4fae05107d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb4af3a0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:23.765922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-599c6ea5-009c-4949-9bb9-ae4fae05107d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.062358 Oct 28 11:01:23.766785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 5/10] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:23 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:01:24.778901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c8924ca6-445e-4af5-b9e5-1f29b74c3d0a tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb4bc0d0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:24.837597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c8924ca6-445e-4af5-b9e5-1f29b74c3d0a tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.061806 Oct 28 11:01:24.839262 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 6/11] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:24 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:01:25.851352 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fef35620-47ba-41b5-8fe4-e9a834c4b7b8 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb4bc790>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:25.935446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fef35620-47ba-41b5-8fe4-e9a834c4b7b8 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.088017 Oct 28 11:01:25.936024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 7/12] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:25 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:01:26.028452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:01:26.029553 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:01:26.030574 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:01:26.031620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:01:26.033017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:01:26.070417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:26.071045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:26.071425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:26.072159 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:26.074454 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:26.074956 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:26.156684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Getting quotas for project 115fe2408cba49bb8b84af24fc8ce604. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:01:26.170975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Getting quotas for user 1903d1aaa3444181ac7b63234f4e8970 and project 115fe2408cba49bb8b84af24fc8ce604. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:01:26.200779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:01:26.209065 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:01:26.403182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-da6af62c-1544-4d6d-9230-7c074f742ad2 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.627826 Oct 28 11:01:26.406032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 6/13] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:01:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 3647 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:01:26.417627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c27bc70f-af92-43b3-816d-e616c3f9ebdf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:26.422176 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c27bc70f-af92-43b3-816d-e616c3f9ebdf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:26.745592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c27bc70f-af92-43b3-816d-e616c3f9ebdf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:26.787916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c27bc70f-af92-43b3-816d-e616c3f9ebdf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1214 microversion: 2.1 time: 0.373849 Oct 28 11:01:26.793724 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 8/14] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:26 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1214 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:26.948855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1d0f7ca6-148c-428d-a1be-97a218d8ab63 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fab6a430>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:27.008469 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1d0f7ca6-148c-428d-a1be-97a218d8ab63 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.063754 Oct 28 11:01:27.009170 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 7/15] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:26 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:27.816314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-822cad30-1cf7-43b6-bd41-6c959490f8e7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:27.818232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-822cad30-1cf7-43b6-bd41-6c959490f8e7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:27.834300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ffa08a9b-000c-45b5-ae10-3f9d5d81b3b7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'create', calling method: .version_select at 0x7f42fb470280>, body: {"keypair": {"name": "tempest-keypair-2010242548", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLGMwvyvGsvBPzpawWCQBfHWAtc0pn+TWWUHgFGCetpGkH8IlxvQxs9w2XjcAfCb/QQEF1ewqk6EcH2g1CxA4yucImBeaa0RLN9Tz4h+m91aWhh8Wjmo++WUdZ+6Bur2Sg=="}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:27.948437 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-ffa08a9b-000c-45b5-ae10-3f9d5d81b3b7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for project 312745d2566343faa50fa7c6f06483f1. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:01:27.963756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-ffa08a9b-000c-45b5-ae10-3f9d5d81b3b7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for user fc81231c965947ecaced8013076edafc and project 312745d2566343faa50fa7c6f06483f1. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:01:28.043925 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-822cad30-1cf7-43b6-bd41-6c959490f8e7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:28.072510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ffa08a9b-000c-45b5-ae10-3f9d5d81b3b7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.486365 Oct 28 11:01:28.073122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 9/16] 10.222.0.46 () {62 vars in 1200 bytes} [Fri Oct 28 11:01:27 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 488 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 28 11:01:28.081373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-822cad30-1cf7-43b6-bd41-6c959490f8e7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1214 microversion: 2.1 time: 0.269839 Oct 28 11:01:28.084800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1ae2f8cf-3893-451e-b715-8b336dbebbfb tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb3f1a60>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:28.100387 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 8/17] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:27 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1214 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:28.151244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1ae2f8cf-3893-451e-b715-8b336dbebbfb tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.077058 Oct 28 11:01:28.151813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 10/18] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:28 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:28.881653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1288875069", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:28.999915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] validate_networks() for [('57d99b3c-83bf-49e2-81d2-c12ed26cdb71', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:01:29.108028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:29.109095 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:29.123546 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:29.124028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:29.124483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:29.125210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:29.126835 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:29.128874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:29.481284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:29.508428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-94e58d5d-f927-4d23-b2b1-71c5226e94eb tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1266 microversion: 2.1 time: 0.404229 Oct 28 11:01:29.512423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 11/19] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:29 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1266 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:29.524783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-43a5e085-dcf5-4c9c-972d-817eeeed03d7 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb324ee0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:29.606346 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-43a5e085-dcf5-4c9c-972d-817eeeed03d7 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 288 microversion: 2.1 time: 0.091364 Oct 28 11:01:29.606903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 12/20] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:29 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 288 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:29.620043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-323ab73d-52f7-4b61-9038-182284e670e2 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb330160>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:29.841669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-323ab73d-52f7-4b61-9038-182284e670e2 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "DELETE /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 202 len: 0 microversion: 2.1 time: 0.226930 Oct 28 11:01:29.842524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 13/21] 10.222.0.46 () {58 vars in 1254 bytes} [Fri Oct 28 11:01:29 2022] DELETE /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 0 bytes in 229 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 11:01:29.862794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ee4649be-7e3a-4117-9a60-06797f240fcb tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb343c10>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:29.996615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ee4649be-7e3a-4117-9a60-06797f240fcb tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.145582 Oct 28 11:01:30.005808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 14/22] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:29 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:30.136217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:01:30.136665 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:01:30.137159 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:01:30.137779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:01:30.138386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:01:30.163244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:30.163743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:30.164196 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:30.170700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:30.171448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:30.172057 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:30.201751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for project f1f49235fbe64b79bfef2cf828993c56. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:01:30.206568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for user 347e262907da4a83a197aaf42cecf51d and project f1f49235fbe64b79bfef2cf828993c56. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:01:30.219473 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:01:30.224039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:01:30.351958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2b63fb07-ea1a-4ab5-8447-1d51039deb87 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.610252 Oct 28 11:01:30.352669 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 9/23] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:01:28 2022] POST /compute/v2.1/servers => generated 383 bytes in 1616 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:01:30.365809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-68728f17-1366-4bfe-a0d0-e7fbb94f5172 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:30.367073 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-68728f17-1366-4bfe-a0d0-e7fbb94f5172 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:30.545008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:30.545008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:30.553844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:30.554383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:30.554828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:30.555539 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:30.556000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:30.556426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:30.562285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-68728f17-1366-4bfe-a0d0-e7fbb94f5172 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:30.572617 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-68728f17-1366-4bfe-a0d0-e7fbb94f5172 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1222 microversion: 2.1 time: 0.211313 Oct 28 11:01:30.573373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 15/24] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:30 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1222 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:30.788739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2087455927", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:31.021204 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] validate_networks() for [('57d99b3c-83bf-49e2-81d2-c12ed26cdb71', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:01:31.113670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:31.139841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-24f11fca-45d3-48b8-afad-7f65107dd463 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1274 microversion: 2.1 time: 0.604768 Oct 28 11:01:31.151241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 10/25] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:30 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1274 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:31.158371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7e99a9b3-377e-4bc4-b62a-6363f79b8d49 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42faaa5f70>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:31.223404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7e99a9b3-377e-4bc4-b62a-6363f79b8d49 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.070296 Oct 28 11:01:31.224898 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 11/26] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:31 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:31.997291 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-106346859", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "95b80201-612c-4027-9944-5fd26540f482"}], "security_groups": [{"name": "tempest-securitygroup--674383477"}], "key_name": "tempest-keypair-2010242548", "user_data": "}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:32.284661 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] validate_networks() for [('95b80201-612c-4027-9944-5fd26540f482', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:01:32.664501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:01:32.665013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:01:32.665387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:01:32.665991 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:01:32.666696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:01:32.678873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:32.679851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:32.680485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:32.681271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:32.681642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:32.682086 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:32.712276 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for project f1f49235fbe64b79bfef2cf828993c56. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:01:32.716346 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for user 347e262907da4a83a197aaf42cecf51d and project f1f49235fbe64b79bfef2cf828993c56. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:01:32.737172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:01:32.744878 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:01:32.833615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-99c9dd68-3a14-4149-93ff-e5fc0144bfed tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.050442 Oct 28 11:01:32.834784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 16/27] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:01:30 2022] POST /compute/v2.1/servers => generated 383 bytes in 2052 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 11:01:32.840667 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:32.841266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:32.849616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:32.850166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:32.850766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:32.851479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:32.851924 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:32.852343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:32.966138 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:32.982192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5328210b-ea36-4eda-8ad6-aa3c92834ba6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1284 microversion: 2.1 time: 0.145336 Oct 28 11:01:32.983122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 17/28] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:32 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1284 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:32.991200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-180bc6b8-d8d8-4de2-a914-80e23734d55f tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb270670>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:33.000091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:01:33.000838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:01:33.001790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:01:33.002509 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:01:33.003427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:01:33.030946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:33.033156 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:33.034916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:33.038279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:33.039606 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:33.040624 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:33.051202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-180bc6b8-d8d8-4de2-a914-80e23734d55f tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.066354 Oct 28 11:01:33.051767 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 18/29] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:32 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:33.056387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0b92b7b5-f7bc-4d24-84a5-513ae9ba4ea0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:33.057189 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0b92b7b5-f7bc-4d24-84a5-513ae9ba4ea0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:33.071262 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for project 312745d2566343faa50fa7c6f06483f1. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:01:33.078394 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for user fc81231c965947ecaced8013076edafc and project 312745d2566343faa50fa7c6f06483f1. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:01:33.125489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:01:33.129359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:01:33.150696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0b92b7b5-f7bc-4d24-84a5-513ae9ba4ea0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:33.158684 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0b92b7b5-f7bc-4d24-84a5-513ae9ba4ea0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1222 microversion: 2.1 time: 0.105682 Oct 28 11:01:33.159266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 19/30] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:33 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1222 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:33.238080 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0ff2b37a-f746-405e-847f-809e40d268b5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.249780 Oct 28 11:01:33.242086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 12/31] 10.222.0.46 () {62 vars in 1188 bytes} [Fri Oct 28 11:01:31 2022] POST /compute/v2.1/servers => generated 401 bytes in 1251 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Oct 28 11:01:33.274201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1a2ff525-70ba-40ef-8bba-bad90a26e7fa tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:33.275660 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1a2ff525-70ba-40ef-8bba-bad90a26e7fa tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:33.348067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1a2ff525-70ba-40ef-8bba-bad90a26e7fa tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:33.356477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1a2ff525-70ba-40ef-8bba-bad90a26e7fa tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:33.357935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1a2ff525-70ba-40ef-8bba-bad90a26e7fa tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:33.366911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1a2ff525-70ba-40ef-8bba-bad90a26e7fa tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1281 microversion: 2.20 time: 0.111260 Oct 28 11:01:33.367526 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 20/32] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:33 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1281 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:34.015344 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:34.017109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:34.045644 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:34.048686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:34.061112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:34.066062 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:34.066641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:34.067091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:34.067882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-31962870-9f89-484c-93d3-134ca8e58663 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb270dc0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:34.144150 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-31962870-9f89-484c-93d3-134ca8e58663 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 200 len: 287 microversion: 2.1 time: 0.083507 Oct 28 11:01:34.144845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 21/33] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:34 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 287 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:34.187209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:34.189703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:34.211078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:34.211614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:34.215822 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:34.216613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:34.217347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:34.218194 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:34.264118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:34.283245 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-33a5499f-fab7-42b7-a5c0-201502c18b24 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1272 microversion: 2.1 time: 0.278938 Oct 28 11:01:34.284669 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 13/34] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:34 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1272 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:34.395651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:34.396279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:34.407336 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:34.407840 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:34.411867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:34.412792 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:34.413573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:34.414247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:34.453119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:34.465949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0106cb1f-0cdf-4aeb-93c4-92517f5dad1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1218 microversion: 2.1 time: 0.282790 Oct 28 11:01:34.466614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 22/35] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:34 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1218 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:34.525795 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:34.536365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:34.538176 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:34.539391 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c3f7d284-9346-4e34-9ac2-83507eefe5e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.148835 Oct 28 11:01:34.540454 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 14/36] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:34 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:35.157243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ca0eaac4-189b-49c1-b4d1-6f5162eb83c3 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb287f70>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:35.242332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-ca0eaac4-189b-49c1-b4d1-6f5162eb83c3 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] HTTP exception thrown: Snapshot f01d69f8-7bcd-4f08-82dd-2a6afa26070f could not be found. Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ca0eaac4-189b-49c1-b4d1-6f5162eb83c3 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Request method failure captured: Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f HTTP/1.1 Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7Z6lprNPMW6ILnH2J3URLeYWe_ea8PbNyePxXaTFP1wAJoa0VsZPmKb3SSpGszejG6yQNqjxxzkdL0jO9V2RovtGOBADoQVj-YBU1Mjk3lm9LNl54SfHZyhzkkFSPrFZzS3a-uEAZDjyI3fOLpCVJT9Rdcw-mrb5IXtmytCzfptOr8 Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:01:35.259385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2adeeab71bd444d290f405908fb221fd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 0fe86aa27b1741e38732e0e97834a4f5 Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Snapshot f01d69f8-7bcd-4f08-82dd-2a6afa26070f could not be found. Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'f01d69f8-7bcd-4f08-82dd-2a6afa26070f'} Oct 28 11:01:35.260921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot f01d69f8-7bcd-4f08-82dd-2a6afa26070f could not be found. (HTTP 404) (Request-ID: req-6899064b-3d78-4395-9449-4b4809de538f) Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:35.261837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:01:35.263067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot f01d69f8-7bcd-4f08-82dd-2a6afa26070f could not be found. Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 28 11:01:35.264236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot f01d69f8-7bcd-4f08-82dd-2a6afa26070f could not be found. Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot f01d69f8-7bcd-4f08-82dd-2a6afa26070f could not be found. Oct 28 11:01:35.265421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:35.275223 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ca0eaac4-189b-49c1-b4d1-6f5162eb83c3 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Returning 404 to user: Snapshot f01d69f8-7bcd-4f08-82dd-2a6afa26070f could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:01:35.276104 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ca0eaac4-189b-49c1-b4d1-6f5162eb83c3 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f" status: 404 len: 111 microversion: 2.1 time: 0.123056 Oct 28 11:01:35.281807 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 23/37] 10.222.0.46 () {58 vars in 1251 bytes} [Fri Oct 28 11:01:35 2022] GET /compute/v2.1/os-snapshots/f01d69f8-7bcd-4f08-82dd-2a6afa26070f => generated 111 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:01:35.286347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b7f40084-c33e-4dc4-9adb-0ee1f22a422d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42faa8cee0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:35.312903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:35.313763 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:35.327946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:35.330453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:35.331069 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:35.334217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:35.336905 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:35.337524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:35.527926 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b7f40084-c33e-4dc4-9adb-0ee1f22a422d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "DELETE /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 202 len: 0 microversion: 2.1 time: 0.246665 Oct 28 11:01:35.529880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 15/38] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:01:35 2022] DELETE /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 0 bytes in 249 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:01:35.535365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:35.536755 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:35.561502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:35.563169 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:35.571470 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:35.576978 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:35.578605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:35.580590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:35.798323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:35.811158 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-33bccf4b-5498-4a99-9702-e28e949fdbd3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.279947 Oct 28 11:01:35.813077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 16/39] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:35 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:35.827615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-182d4c2d-5364-4323-ba70-3cc408d6aaa6 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fab7f9d0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:35.963387 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-182d4c2d-5364-4323-ba70-3cc408d6aaa6 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 200 len: 345 microversion: 2.1 time: 0.148045 Oct 28 11:01:35.964870 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 17/40] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:35 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 345 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:35.969569 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:35.971102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:35.978599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:35.979459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:35.980175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:35.981037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:35.981658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:35.982271 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:36.062877 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:36.074723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:36.076306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:36.077334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1e6b78e0-6ad2-49d6-bf51-d1c468124a0b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.111379 Oct 28 11:01:36.077899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 18/41] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:35 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:36.083509 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2988b3ae-ca51-4705-9310-e26413e01394 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42faa2adc0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:36.152289 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2988b3ae-ca51-4705-9310-e26413e01394 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/70f67492-0ce3-4b55-98d1-fd317ae3c1ed" status: 200 len: 350 microversion: 2.1 time: 0.073069 Oct 28 11:01:36.153003 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 19/42] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:36 2022] GET /compute/v2.1/os-volumes/70f67492-0ce3-4b55-98d1-fd317ae3c1ed => generated 350 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:36.549119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:36.560179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-03ec6721-5700-4d6b-9a2e-fe049439944b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 1.251928 Oct 28 11:01:36.563926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 24/43] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:35 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 1256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:36.841148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:36.841148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:36.849269 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:36.850144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:36.850694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:36.851506 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:36.852104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:36.852631 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:36.956658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:36.971816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dc07ba51-c103-49f9-a2e0-cf154ceaa6d7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.135844 Oct 28 11:01:36.982643 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 20/44] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:36 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:36.993360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dc5edb57-123c-4294-bbe1-f4e06bf9100d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fb284a60>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:37.048076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dc5edb57-123c-4294-bbe1-f4e06bf9100d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 200 len: 345 microversion: 2.1 time: 0.063691 Oct 28 11:01:37.048651 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 25/45] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:36 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 345 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:37.105584 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:37.106315 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:37.113543 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:37.114223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:37.115037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:37.116336 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:37.117157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:37.118212 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:37.201593 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:37.210310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:37.211768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:37.212916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9a746a79-6a73-4d2e-a667-359a71481cda tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.111831 Oct 28 11:01:37.213424 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 21/46] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:37 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:37.587729 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:37.588874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:37.598147 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:37.598652 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:37.602463 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:37.603262 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:37.603752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:37.604196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:37.769837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:37.778943 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e11f68e0-abd8-4a76-ab2e-18174d2ba54c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.194633 Oct 28 11:01:37.781383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 26/47] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:37 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:37.846064 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-1c03ee06-74fb-4867-8c41-84e221699df3 req-f93783cf-ff57-4b37-9daf-81d5087188a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bd2715aa-33ac-4197-90f4-d2dc79a7f07e", "tag": "0da3d13b-05f9-481e-8125-00a60da3dc59"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:37.859137 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1c03ee06-74fb-4867-8c41-84e221699df3 req-f93783cf-ff57-4b37-9daf-81d5087188a4 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:37.859608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1c03ee06-74fb-4867-8c41-84e221699df3 req-f93783cf-ff57-4b37-9daf-81d5087188a4 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:37.860030 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1c03ee06-74fb-4867-8c41-84e221699df3 req-f93783cf-ff57-4b37-9daf-81d5087188a4 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:37.881342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-1c03ee06-74fb-4867-8c41-84e221699df3 req-f93783cf-ff57-4b37-9daf-81d5087188a4 service nova] Creating event network-changed:0da3d13b-05f9-481e-8125-00a60da3dc59 for instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e on hv-cinder-zed-3 Oct 28 11:01:37.882265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1c03ee06-74fb-4867-8c41-84e221699df3 req-f93783cf-ff57-4b37-9daf-81d5087188a4 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120544) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 28 11:01:37.882804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1c03ee06-74fb-4867-8c41-84e221699df3 req-f93783cf-ff57-4b37-9daf-81d5087188a4 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120544) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 28 11:01:37.884396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:37.884854 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:37.885276 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:37.886037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:37.886438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:37.886854 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:37.896147 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.compute.rpcapi [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 28 11:01:37.897244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] Releasing lock "compute-rpcapi-router" {{(pid=120544) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 28 11:01:37.915626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2426b107-a8ea-446c-8195-63093cedc72d None None] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.220008 Oct 28 11:01:37.916422 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 22/48] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:01:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:01:37.996087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:37.996704 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:38.004291 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:38.004847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:38.005352 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:38.006182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:38.006715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:38.007201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:38.062300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1aa5060e-1fec-4eae-9f68-d619c6896c0c tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42fab1baf0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:38.069534 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:38.088826 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-808f9f54-4ab1-4dc8-9c12-b856ae4562e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.096637 Oct 28 11:01:38.101630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 27/49] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:37 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:38.124443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1aa5060e-1fec-4eae-9f68-d619c6896c0c tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 200 len: 345 microversion: 2.1 time: 0.065668 Oct 28 11:01:38.125068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 23/50] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:38 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 345 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:01:38.238598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:38.239214 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:38.245247 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:38.245815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:38.246363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:38.247136 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:38.247668 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:38.248198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:38.311897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:38.320468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:38.321939 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:38.323070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-44a4c2f2-e4ef-4a96-8149-51d3aed44623 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.088641 Oct 28 11:01:38.323628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 28/51] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:38 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:38.804330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:38.805094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:38.811878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:38.812365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:38.812810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:38.813512 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:38.813965 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:38.814409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:38.968451 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:38.976370 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7c8a774e-0d39-49a2-8bab-f39c5ff968ff tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.176795 Oct 28 11:01:38.977039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 24/52] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:38 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:39.118764 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:39.120099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:39.127714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:39.128517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:39.130320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:39.131385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:39.132094 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:39.132744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:39.142673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-16f9e375-78a9-4ca2-9a2f-66230bb95f8d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42faad29d0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:39.215838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-16f9e375-78a9-4ca2-9a2f-66230bb95f8d tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 200 len: 345 microversion: 2.1 time: 0.078356 Oct 28 11:01:39.216592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 25/53] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:39 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 345 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:39.235256 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:39.244271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-09037834-2493-43ea-90f2-0becb57e093c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.130640 Oct 28 11:01:39.263300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 29/54] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:39 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:39.351874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:39.352496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:39.359027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:39.359855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:39.360666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:39.362951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:39.364218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:39.364880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:39.442652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:39.450948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:39.452760 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:39.454398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-05fc4857-f33e-4a01-ad58-53c0dad29ec8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.107188 Oct 28 11:01:39.455131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 26/55] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:39 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:40.000362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:40.001094 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:40.015555 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:40.016054 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:40.016741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:40.017583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:40.018067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:40.018489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:40.232303 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7014acbd-ced1-4b52-adb4-9c86e984ffa8 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42faa8cd30>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:40.352519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:40.366955 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-aa05d3b8-d1fc-49dd-a4db-99ef667237c5 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.368977 Oct 28 11:01:40.368783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 30/56] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:39 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:40.377787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:40.378444 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:40.388557 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:40.388840 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7014acbd-ced1-4b52-adb4-9c86e984ffa8 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 200 len: 345 microversion: 2.1 time: 0.160774 Oct 28 11:01:40.389564 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:40.389771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 27/57] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:40 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 345 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:40.393124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:40.394572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:40.395159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:40.396918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:40.482846 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:40.484265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:40.502831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:40.503041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:40.506300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:40.507984 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:40.509493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:40.511159 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:40.513132 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:40.525631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ff8d92b5-404c-4867-b7a5-ba28c8f4dd1e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.154448 Oct 28 11:01:40.529867 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 31/58] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:40 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:40.682570 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:40.700896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:40.706983 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:40.712279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b8b105fd-5465-42e8-9647-ba668410fc7c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.234962 Oct 28 11:01:40.716046 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 28/59] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:40 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:40.830401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b071cc10-4456-45c3-92a4-4f8e2bb37f65 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42fb2848b0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:40.902166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b071cc10-4456-45c3-92a4-4f8e2bb37f65 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/9c07e52a-8600-4f37-b194-0264547cb29d" status: 200 len: 351 microversion: 2.1 time: 0.082554 Oct 28 11:01:40.902735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 32/60] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:40 2022] GET /compute/v2.1/os-volumes/9c07e52a-8600-4f37-b194-0264547cb29d => generated 351 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:40.918777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8d5803e1-e4fe-409f-86df-fd9a0fe742b3 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42fab1bb80>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.040558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8d5803e1-e4fe-409f-86df-fd9a0fe742b3 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.126536 Oct 28 11:01:41.041215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 29/61] 10.222.0.46 () {58 vars in 1134 bytes} [Fri Oct 28 11:01:40 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:41.057698 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-66f5de4e-f6d7-4ffe-b042-a89c2abdd917 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42fb287dc0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.147201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-66f5de4e-f6d7-4ffe-b042-a89c2abdd917 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.093168 Oct 28 11:01:41.148860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 33/62] 10.222.0.46 () {58 vars in 1149 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:41.164116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a4fe7d9c-d12e-419b-95e0-91e4c8e30b07 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42faa8c040>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.274066 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a4fe7d9c-d12e-419b-95e0-91e4c8e30b07 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.113907 Oct 28 11:01:41.275417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 30/63] 10.222.0.46 () {58 vars in 1134 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:41.289621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4bb686bb-2adb-4b59-b8e7-1642e58e7c73 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42faa29550>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.393835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.395541 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:41.412445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:41.414148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:41.414701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:41.416099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:41.417956 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:41.418485 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:41.421187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4bb686bb-2adb-4b59-b8e7-1642e58e7c73 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.135933 Oct 28 11:01:41.422657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 34/64] 10.222.0.46 () {58 vars in 1167 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:41.434697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9d89588a-30dd-41eb-aff9-254bae1d6e69 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42faa0e310>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.483028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-9d89588a-30dd-41eb-aff9-254bae1d6e69 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] HTTP exception thrown: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9d89588a-30dd-41eb-aff9-254bae1d6e69 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Request method failure captured: Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 HTTP/1.1 Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7Z6lprNPMW6ILnH2J3URLeYWe_ea8PbNyePxXaTFP1wAJoa0VsZPmKb3SSpGszejG6yQNqjxxzkdL0jO9V2RovtGOBADoQVj-YBU1Mjk3lm9LNl54SfHZyhzkkFSPrFZzS3a-uEAZDjyI3fOLpCVJT9Rdcw-mrb5IXtmytCzfptOr8 Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:01:41.485601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2adeeab71bd444d290f405908fb221fd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 0fe86aa27b1741e38732e0e97834a4f5 Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'ae826d88-35f4-42d4-9353-c1b3f5e95bb9'} Oct 28 11:01:41.487975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. (HTTP 404) (Request-ID: req-205e63f9-f8c9-4cac-b2a5-938fccbb440b) Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.489088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:01:41.490687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 11:01:41.491731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9d89588a-30dd-41eb-aff9-254bae1d6e69 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Returning 404 to user: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9d89588a-30dd-41eb-aff9-254bae1d6e69 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 404 len: 109 microversion: 2.1 time: 0.064323 Oct 28 11:01:41.492643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 35/65] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 109 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:01:41.500624 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-eeee881e-2720-4d9f-99fd-2ddf58fd0af9 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42fb287a60>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.594867 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-eeee881e-2720-4d9f-99fd-2ddf58fd0af9 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.105251 Oct 28 11:01:41.595483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 36/66] 10.222.0.46 () {58 vars in 1170 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:41.609756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c01e6d0a-eb93-4d84-bdb6-c6988677f50b tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42faa0e0d0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.640218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-c01e6d0a-eb93-4d84-bdb6-c6988677f50b tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] HTTP exception thrown: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c01e6d0a-eb93-4d84-bdb6-c6988677f50b tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Request method failure captured: Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 HTTP/1.1 Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7Z6lprNPMW6ILnH2J3URLeYWe_ea8PbNyePxXaTFP1wAJoa0VsZPmKb3SSpGszejG6yQNqjxxzkdL0jO9V2RovtGOBADoQVj-YBU1Mjk3lm9LNl54SfHZyhzkkFSPrFZzS3a-uEAZDjyI3fOLpCVJT9Rdcw-mrb5IXtmytCzfptOr8 Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:01:41.643267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2adeeab71bd444d290f405908fb221fd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 0fe86aa27b1741e38732e0e97834a4f5 Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'ae826d88-35f4-42d4-9353-c1b3f5e95bb9'} Oct 28 11:01:41.644460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. (HTTP 404) (Request-ID: req-7a6bcba1-2881-4112-8949-265a623c3c0f) Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.645629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 11:01:41.646665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 28 11:01:41.647604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.650770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.652207 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c01e6d0a-eb93-4d84-bdb6-c6988677f50b tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Returning 404 to user: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:01:41.653265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c01e6d0a-eb93-4d84-bdb6-c6988677f50b tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "DELETE /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 404 len: 109 microversion: 2.1 time: 0.056762 Oct 28 11:01:41.654315 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 37/67] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:01:41 2022] DELETE /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 109 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:01:41.659101 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.660579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:41.668299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:41.669139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:41.669565 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:41.670447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:41.671429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:41.672187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:41.672815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:41.679423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e80e75ee-a071-4253-a0e1-d6df53111e99 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.290331 Oct 28 11:01:41.681144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 31/68] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:41.703950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cd3cb9d0-ca52-4656-a552-d28f860fc73d tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42fab1bf70>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.818872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:41.828945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-54735380-ab0e-43c0-8d08-a06c8c6f4b18 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.173480 Oct 28 11:01:41.829558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 38/69] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:41.843945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-cd3cb9d0-ca52-4656-a552-d28f860fc73d tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.161552 Oct 28 11:01:41.844512 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 32/70] 10.222.0.46 () {58 vars in 1155 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:41.853075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.853075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:41.864896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:41.865415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:41.866007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:41.867132 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:41.867735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:41.868078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9291f9e1-c4df-462f-9c80-6b9192a06518 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Calling method '.version_select at 0x7f42faa0e670>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.868203 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:41.902655 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-9291f9e1-c4df-462f-9c80-6b9192a06518 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] HTTP exception thrown: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9291f9e1-c4df-462f-9c80-6b9192a06518 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Request method failure captured: Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 HTTP/1.1 Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7Z6lprNPMW6ILnH2J3URLeYWe_ea8PbNyePxXaTFP1wAJoa0VsZPmKb3SSpGszejG6yQNqjxxzkdL0jO9V2RovtGOBADoQVj-YBU1Mjk3lm9LNl54SfHZyhzkkFSPrFZzS3a-uEAZDjyI3fOLpCVJT9Rdcw-mrb5IXtmytCzfptOr8 Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:01:41.904571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_2adeeab71bd444d290f405908fb221fd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/2adeeab71bd444d290f405908fb221fd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 2adeeab71bd444d290f405908fb221fd Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-113260461 Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 0fe86aa27b1741e38732e0e97834a4f5 Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesSnapshotsTestJSON-113260461-project Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'ae826d88-35f4-42d4-9353-c1b3f5e95bb9'} Oct 28 11:01:41.905672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. (HTTP 404) (Request-ID: req-a9421d9c-23ee-4b45-8a26-686b12ce0cab) Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.906419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:01:41.907849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 11:01:41.908813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9291f9e1-c4df-462f-9c80-6b9192a06518 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] Returning 404 to user: Volume ae826d88-35f4-42d4-9353-c1b3f5e95bb9 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9291f9e1-c4df-462f-9c80-6b9192a06518 tempest-VolumesSnapshotsTestJSON-113260461 tempest-VolumesSnapshotsTestJSON-113260461-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9" status: 404 len: 109 microversion: 2.1 time: 0.056368 Oct 28 11:01:41.910901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 39/71] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes/ae826d88-35f4-42d4-9353-c1b3f5e95bb9 => generated 109 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:01:41.920006 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b2e39535-79c8-4f80-a94d-7e96fde8f494 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42fb2848b0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:41.986845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:41.998134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:42.003403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:42.005470 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-181105bd-a77e-433e-9302-0445e503ed3c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.159967 Oct 28 11:01:42.006058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 33/72] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:42.039754 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b2e39535-79c8-4f80-a94d-7e96fde8f494 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.129601 Oct 28 11:01:42.040393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 40/73] 10.222.0.46 () {58 vars in 1188 bytes} [Fri Oct 28 11:01:41 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:42.054232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ef01c151-94a3-43d6-8191-5315ea1345d2 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] Calling method '.version_select at 0x7f42faad2940>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:42.142127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ef01c151-94a3-43d6-8191-5315ea1345d2 tempest-VolumesTestJSON-1846441092 tempest-VolumesTestJSON-1846441092-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.092218 Oct 28 11:01:42.142756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 34/74] 10.222.0.46 () {58 vars in 1155 bytes} [Fri Oct 28 11:01:42 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:42.711023 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:42.721116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:42.722191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:42.736968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:42.737728 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:42.739825 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:42.748978 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:42.749568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:42.750311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:42.856085 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:42.856627 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:42.869871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:42.870371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:42.870804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:42.871501 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:42.871951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:42.876046 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:42.924897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:42.937175 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-33d75d32-cfeb-4988-9482-197abc363c76 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.227491 Oct 28 11:01:42.938601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 41/75] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:42 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:42.961307 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:42.970968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0a5c533a-6ad6-4bde-8a54-a10c9b81a211 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.118663 Oct 28 11:01:42.971540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 35/76] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:42 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:43.034471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:43.035205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:43.041607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:43.051636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:43.051636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:43.051636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:43.051636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:43.052341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:43.110372 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:43.118735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:43.120185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:43.121296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-399efda0-0ae2-4c6a-97b0-806ea4b504f6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.095094 Oct 28 11:01:43.121847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 42/77] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:43 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:43.962519 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:43.963131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:43.967565 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=20 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:43.973615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:43.974178 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:43.974859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:43.975615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:43.976388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:43.976647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:43.999620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:44.002793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:44.010616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=18 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:44.015997 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:44.016370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:44.016790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:44.017478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:44.025633 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:44.026175 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:44.168602 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:44.179287 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-72578c2b-f5d8-47a4-b134-680d70e65c79 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.221990 Oct 28 11:01:44.187402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:44.194964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 36/78] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:43 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:44.197710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8dbc8794-d55e-45d0-a384-bfb1232a4ddc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.204152 Oct 28 11:01:44.203013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:44.203878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:44.209089 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 43/79] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:43 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:44.211832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:44.212583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:44.213018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:44.213930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:44.214990 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:44.215623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:44.324338 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:44.328232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:44.334424 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:44.335733 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:44.337704 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-271cf3c2-8c6e-4eab-a51e-439aa126e9f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.141649 Oct 28 11:01:44.346405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 37/80] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:44 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:45.207900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:45.208572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:45.214447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:45.214918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:45.215343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:45.216024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:45.216471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:45.216887 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:45.224573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:45.225213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:45.227326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=9 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:45.232383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:45.232871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:45.233573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:45.234298 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:45.234751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:45.235175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:45.305516 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:45.314208 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-99a96e20-1341-40e7-ab37-009001a5e9bc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.093857 Oct 28 11:01:45.326153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 38/81] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:45 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:45.363947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:45.364562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:45.370757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:45.371535 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:45.372793 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:45.373729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:45.374452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:45.375150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:45.376271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:45.386936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-76397377-5f2f-4cd7-8fcf-d1361114ebd3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.182791 Oct 28 11:01:45.388579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 44/82] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:45 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:45.448976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:45.457937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:45.459194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:45.460308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-78ca7333-60cb-4dbb-b189-0560dbe8776f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.100829 Oct 28 11:01:45.460872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 39/83] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:45 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:46.339384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:46.340019 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:46.345863 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:46.346364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:46.346790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:46.348095 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:46.348543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:46.348956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:46.408765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:46.410914 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:46.411624 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:46.417191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cd44b5fd-721d-4fd9-81d7-0e4c9c9ee504 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.081388 Oct 28 11:01:46.417550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 45/84] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:46 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:46.422199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:46.422686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:46.423122 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:46.423966 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:46.424439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:46.424851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:46.486420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:46.487040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:46.492895 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:46.493361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:46.493784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:46.494489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:46.494930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:46.495342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:46.499508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:46.565543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:46.573652 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:46.574044 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:46.575138 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:46.576231 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7f661cd6-dfdf-4fd2-9104-f8173a42f56f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.094045 Oct 28 11:01:46.576793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 46/85] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:46 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:01:46.581617 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9a224454-ccba-41c5-bdf3-9e8fdd948991 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.175395 Oct 28 11:01:46.582746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 40/86] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:46 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:47.440144 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:47.442143 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:47.449016 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:47.449783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:47.450506 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:47.451564 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:47.452309 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:47.453476 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:47.532172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:47.542219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-13a6a28a-c245-4644-85cf-6bcd8d08f5a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.106903 Oct 28 11:01:47.552007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 47/87] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:47 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:47.604833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:47.605451 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:47.606111 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:47.606722 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:47.612024 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:47.612729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:47.613757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:47.614489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:47.615269 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:47.615717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:47.619100 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:47.619983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:47.620741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:47.621532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:47.625589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:47.626190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:47.708265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:47.716928 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:47.718390 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:47.719867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-034afd80-7d37-4a02-b11d-36a286ca9b41 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.121006 Oct 28 11:01:47.720690 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 41/88] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:47 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:47.786347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:47.793911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-283d956a-e0ed-48c1-92ef-ff9ca2baa57d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.191997 Oct 28 11:01:47.794497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 48/89] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:47 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:48.563376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=8 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:48.567748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:48.568690 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:48.575962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:48.576662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:48.577386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:48.578144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:48.578588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:48.579013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:48.681931 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:48.698429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fa062ecd-b0e3-48c4-8794-8a64c9bb8946 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.135611 Oct 28 11:01:48.701116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 42/90] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:48 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:48.753195 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:48.754574 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:48.770323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:48.770816 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:48.771244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:48.772305 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:48.772756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:48.773177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:48.819499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:48.820157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:48.845450 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:48.845944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:48.849502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:48.850307 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:48.851032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:48.855292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:48.885896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:48.898640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:48.900747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:48.902265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-828f48a5-a05d-45b0-b229-f637178f667c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.155714 Oct 28 11:01:48.908012 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 49/91] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:48 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:49.046331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:49.056085 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a3cc29a9-88fa-4361-8a4c-4350c23ffb81 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.241784 Oct 28 11:01:49.057256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 43/92] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:48 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:49.721984 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:49.723294 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:49.726351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:49.729847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:49.730856 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:49.731786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:49.733580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:49.734332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:49.734752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:49.806508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:49.827208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d1c93738-af37-49ba-a3c0-855b10398ad8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.108148 Oct 28 11:01:49.832331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 50/93] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:49 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:49.935283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:49.935941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:49.944981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:49.945480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:49.945916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:49.946661 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:49.947106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:49.947528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:50.053285 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:50.063147 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:50.065027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:50.067060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8e70b8be-e742-4a1b-a3e9-03644339307b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.138485 Oct 28 11:01:50.067942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 44/94] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:49 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:50.080560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:50.081170 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:50.089896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:50.090420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:50.090859 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:50.091550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:50.092001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:50.092415 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:50.341811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:50.353216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b51bf885-251a-427a-869a-2c6622888ff0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.273084 Oct 28 11:01:50.363385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 51/95] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:50 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:50.852688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:50.853297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:50.859482 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:50.859962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:50.860390 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:50.861071 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:50.862049 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:50.862049 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:50.922983 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:50.930569 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-95e8ab4e-a4a6-4aef-a345-6ad4f7abbb26 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.081697 Oct 28 11:01:50.931145 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 45/96] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:50 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:51.101617 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:51.102187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:51.115985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:51.116468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:51.116895 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:51.117581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:51.118059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:51.118479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:51.185843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:51.193716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:51.195177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:51.196260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ab515fea-9750-4c4b-bcd9-8c35cc2bf325 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.099745 Oct 28 11:01:51.196808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 52/97] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:51 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:51.375133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:51.376436 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:51.382994 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:51.383475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:51.383907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:51.384621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:51.385417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:51.385907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:51.533523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:51.540970 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-75d54f31-f253-48d1-b58a-bc9a9c9da751 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.170151 Oct 28 11:01:51.541531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 46/98] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:51 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:51.952620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:51.953233 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:51.959464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:51.959945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:51.960378 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:51.961068 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:51.961523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:51.961943 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:52.025491 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:52.033301 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d0790a87-c6e1-4595-acb7-3120b3e07ab3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.084481 Oct 28 11:01:52.033860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 53/99] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:51 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:52.223577 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:52.224787 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:52.230794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:52.231263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:52.231884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:52.232640 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:52.233205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:52.234040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:52.298616 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:52.307303 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:52.308735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:52.309835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a775f95f-6417-41fb-97c8-f9ceec2b7e4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.090871 Oct 28 11:01:52.310419 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 47/100] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:52 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:52.564191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:52.567153 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:52.574854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:52.576036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:52.576626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:52.577683 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:52.578178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:52.578607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:52.713904 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:52.721803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-78126d46-0067-4545-b796-1869907e536e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.161626 Oct 28 11:01:52.723072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 54/101] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:52 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:53.056694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:53.057559 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:53.063781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:53.064338 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:53.064839 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:53.065601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:53.066678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:53.067155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:53.129400 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:53.136890 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1ff4b663-ef1e-49c9-9a3e-d25bd69db89d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.084410 Oct 28 11:01:53.137452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 48/102] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:53 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:53.335078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:53.335678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:53.341600 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:53.342106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:53.342536 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:53.343208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:53.343653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:53.344065 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:53.402782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:53.410999 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:53.412466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:53.413558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-73e47cd4-575c-4cc7-be43-43c532993fef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.082427 Oct 28 11:01:53.414212 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 55/103] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:53 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:01:53.745189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:53.745804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:53.751691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:53.752170 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:53.752637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:53.753316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:53.753815 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:53.754268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:53.877217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:53.884780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5d27afd6-3f4d-41fb-a33e-b3cd6ccfa7a7 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.143428 Oct 28 11:01:53.885342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 49/104] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:53 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:54.159032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:54.160244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:54.166286 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:54.166767 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:54.167200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:54.167881 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:54.168337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:54.168764 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:54.230222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:54.238428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-eb1fa478-798d-4429-a468-166022a97f5f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1274 microversion: 2.1 time: 0.083144 Oct 28 11:01:54.238996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 56/105] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:54 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1274 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:54.438828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:54.439364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:54.445808 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:54.446312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:54.446741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:54.447407 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:54.447855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:54.448267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:54.509608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:54.517803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:54.519923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:54.521024 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dcbba6b0-8dd0-4a27-9da5-cb0b40cfec40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1277 microversion: 2.20 time: 0.086185 Oct 28 11:01:54.521566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 50/106] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:54 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1277 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:54.908643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:54.909363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:54.915491 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:54.915965 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:54.916392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:54.917079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:54.917524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:54.917938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:55.262791 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:55.263788 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:55.282287 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:55.283249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:55.286187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:55.287161 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:55.290752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:55.291787 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:55.352697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:55.365764 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-60561f0c-034d-4a82-8dad-c433d7626dc3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.460652 Oct 28 11:01:55.367324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 57/107] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:54 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:55.465479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:55.475882 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-932c0e22-8407-4116-8a5c-151356b1c260 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1292 microversion: 2.1 time: 0.218015 Oct 28 11:01:55.476921 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 51/108] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:55 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1292 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:01:55.545875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:55.546728 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:55.553695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:55.554316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:55.555045 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:55.556014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:55.556800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:55.557188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:55.667683 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:55.683828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:55.686035 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:55.687204 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a3f9982d-0788-410e-b721-f9e40ee4c0b6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1333 microversion: 2.20 time: 0.145479 Oct 28 11:01:55.687788 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 58/109] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:55 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1333 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:55.904963 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-6f4f920a-e105-4a1d-8a2a-1e56c0bef591 req-a9fad64b-fbbd-4d55-945f-6d1fc0042a1a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54cf39f2-cfd4-4f12-bddb-36666db9e1d5", "tag": "9cdeceb0-c9f5-42e0-9e0e-62d116b04c2f"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:55.916700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-6f4f920a-e105-4a1d-8a2a-1e56c0bef591 req-a9fad64b-fbbd-4d55-945f-6d1fc0042a1a service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:55.917401 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-6f4f920a-e105-4a1d-8a2a-1e56c0bef591 req-a9fad64b-fbbd-4d55-945f-6d1fc0042a1a service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:55.918068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-6f4f920a-e105-4a1d-8a2a-1e56c0bef591 req-a9fad64b-fbbd-4d55-945f-6d1fc0042a1a service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:55.932441 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-6f4f920a-e105-4a1d-8a2a-1e56c0bef591 req-a9fad64b-fbbd-4d55-945f-6d1fc0042a1a service nova] Creating event network-changed:9cdeceb0-c9f5-42e0-9e0e-62d116b04c2f for instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 on hv-cinder-zed-3 Oct 28 11:01:55.938754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-6f4f920a-e105-4a1d-8a2a-1e56c0bef591 req-a9fad64b-fbbd-4d55-945f-6d1fc0042a1a service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037315 Oct 28 11:01:55.939361 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 52/110] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:01:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:56.389928 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:56.390579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:56.406849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:56.407879 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:56.408339 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:56.412399 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:56.414863 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:56.415723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:56.499160 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:56.499881 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:56.511116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:56.511652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:56.512086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:56.512790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:56.513241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:56.513762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:56.702324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:56.711030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7401a587-0819-43e1-ba36-b43f8f06bf12 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.325054 Oct 28 11:01:56.711678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 59/111] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:56 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:56.716245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:56.716918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:56.722604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:56.723145 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:56.723641 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:56.724318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:56.724977 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:56.725579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:56.786572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:56.794932 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:56.796362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:56.798179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-804bf19f-f3b8-465b-a33d-3fcef2cb0741 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1339 microversion: 2.20 time: 0.085486 Oct 28 11:01:56.798734 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 60/112] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:56 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1339 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:01:56.846736 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:56.856691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d9e6e0fa-0bf8-4cc8-8f95-a835235efd21 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.361209 Oct 28 11:01:56.869229 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 53/113] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:56 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:57.479037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-93d09df9-a19d-4b32-a7f8-20be488e7af6 req-832e12c1-248f-41eb-a4b3-0ca83fb80ab3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "643427ec-cf7f-4058-a731-b96412826c2e", "tag": "c012cb5d-de75-4d0c-8ffe-7a84c1ef9d66"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:01:57.491916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-93d09df9-a19d-4b32-a7f8-20be488e7af6 req-832e12c1-248f-41eb-a4b3-0ca83fb80ab3 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.492383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-93d09df9-a19d-4b32-a7f8-20be488e7af6 req-832e12c1-248f-41eb-a4b3-0ca83fb80ab3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.492806 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-93d09df9-a19d-4b32-a7f8-20be488e7af6 req-832e12c1-248f-41eb-a4b3-0ca83fb80ab3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.511287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-93d09df9-a19d-4b32-a7f8-20be488e7af6 req-832e12c1-248f-41eb-a4b3-0ca83fb80ab3 service nova] Creating event network-changed:c012cb5d-de75-4d0c-8ffe-7a84c1ef9d66 for instance 643427ec-cf7f-4058-a731-b96412826c2e on hv-cinder-zed-3 Oct 28 11:01:57.511803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-93d09df9-a19d-4b32-a7f8-20be488e7af6 req-832e12c1-248f-41eb-a4b3-0ca83fb80ab3 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120543) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 28 11:01:57.512188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-93d09df9-a19d-4b32-a7f8-20be488e7af6 req-832e12c1-248f-41eb-a4b3-0ca83fb80ab3 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120543) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 28 11:01:57.513180 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.513631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.514077 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.514779 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.515223 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.515634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.524343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.compute.rpcapi [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 28 11:01:57.525181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] Releasing lock "compute-rpcapi-router" {{(pid=120543) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 28 11:01:57.541037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-329eeb50-d75d-438f-88d3-b4220a0f0215 None None] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.119270 Oct 28 11:01:57.541632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 61/114] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:01:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:01:57.733784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:57.734419 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:57.740994 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.741471 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.741898 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.742601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.743080 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.743721 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.823886 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:57.824581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:57.830925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.831442 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.831882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.832592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.833027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.833450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.868697 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:57.876545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2f0d9e2c-d0e0-43bf-9baf-21418a6544b1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.146710 Oct 28 11:01:57.884843 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 54/115] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:57 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:57.884843 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:57.884843 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:57.890081 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.890867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.891545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.892461 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:57.893072 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:57.893499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:57.899437 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:57.911895 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:57.922917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:57.924433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:57.925401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-055d2287-773b-4254-b9b1-dc4f22bf0d67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1339 microversion: 2.20 time: 0.105852 Oct 28 11:01:57.930336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 62/116] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:57 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1339 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:01:58.038392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:58.045837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-231a819a-722b-4a8c-8dce-00a684e8ef08 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.167492 Oct 28 11:01:58.046424 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 55/117] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:57 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:58.900631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:58.901240 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:58.907036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:58.907512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:58.907944 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:58.908625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:58.909071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:58.909532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:58.913646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:01:58.950941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:58.951552 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:58.958109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:58.958596 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:58.959028 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:58.959713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:58.960783 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:58.961209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:59.045099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:59.047353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:59.063310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e117c5a0-60a6-453a-a2d8-ab9bd0c0206d tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.166370 Oct 28 11:01:59.063970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 63/118] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:58 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:01:59.067949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:59.069491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:59.071021 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2f5aba1e-4f4a-4314-83db-bf2811070c5a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1339 microversion: 2.20 time: 0.123835 Oct 28 11:01:59.082004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:01:59.082004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:01:59.082945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:59.082945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:59.083190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:59.083459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 56/119] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:01:58 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1339 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:01:59.084292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:01:59.084893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:01:59.085335 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:01:59.208175 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:01:59.215597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c111749b-410c-469a-91f4-5000423144b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.146079 Oct 28 11:01:59.216156 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 64/120] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:01:59 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:00.086604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:00.087257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:00.093321 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:00.093812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:00.094282 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:00.094983 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:00.095432 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:00.096031 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:00.098404 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:00.099491 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:00.107780 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:00.109356 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:00.109948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:00.110821 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:00.111399 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:00.112334 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:00.180117 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:00.190544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:00.192551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:00.193655 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5f1cfd6a-9860-4987-a590-ecdfa7add62a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1339 microversion: 2.20 time: 0.099971 Oct 28 11:02:00.194256 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 65/121] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:00 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1339 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:00.233233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:00.238990 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:00.239594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:00.241728 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0eeadd6a-6df4-45b9-b07c-12500be9f3e1 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.158884 Oct 28 11:02:00.242568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 57/122] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:00 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:00.245316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:00.245782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:00.246251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:00.246923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:00.247528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:00.247943 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:00.823103 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:00.830776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a1172142-d478-421f-95a2-943f58810473 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.596250 Oct 28 11:02:00.831710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 66/123] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:00 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:01.219413 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:01.220058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:01.227332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:01.227781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:01.228654 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:01.229535 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:01.230475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:01.231057 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:01.264890 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:01.267240 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:01.279383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:01.280213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:01.281453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:01.282386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:01.282981 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:01.284147 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:01.320471 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:01.330315 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:01.332668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:01.333956 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c731e572-d507-4bdc-bbf7-7ad24e27edb6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1339 microversion: 2.20 time: 0.118635 Oct 28 11:02:01.334966 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 58/124] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:01 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1339 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:01.497022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:01.505466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-af016cda-04f7-44ba-ba1c-35cf26ae3785 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.244594 Oct 28 11:02:01.506063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 67/125] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:01 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:01.857141 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:01.857683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:01.864397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:01.864869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:01.865302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:01.866007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:01.866452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:01.867065 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:02.028824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:02.039098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-66a9cc86-2f49-47c3-b83d-19f67db9dab9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.185842 Oct 28 11:02:02.039672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 59/126] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:01 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:02.360326 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:02.361542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:02.372853 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:02.372853 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:02.374422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:02.375235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:02.375658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:02.378271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:02.534062 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:02.535566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:02.544953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:02.545756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:02.558892 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:02.559717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:02.560199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:02.560630 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:02.624080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:02.648135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:02.649816 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:02.655203 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5cc7543a-0641-4d13-b52e-39f97598d769 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.299049 Oct 28 11:02:02.655797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 68/127] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:02 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:02.852828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:02.863294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0de74f9a-9b52-4d54-8785-1444f63e24e6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.336876 Oct 28 11:02:02.864300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 60/128] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:02 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:03.063577 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:03.064192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:03.070676 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:03.071154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:03.071583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:03.072269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:03.072732 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:03.073130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:03.244361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:03.255102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6da78b78-07c5-4607-8a36-0d1a385365f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.196003 Oct 28 11:02:03.256258 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 69/129] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:03 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:03.682628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:03.684196 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:03.693535 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:03.694257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:03.695448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:03.696342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:03.697002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:03.697647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:03.840444 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:03.850332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:03.851768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:03.852897 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8f364336-7119-4cd8-abf9-5f8223156283 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.175865 Oct 28 11:02:03.853658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 61/130] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:03 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:03.890922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:03.892775 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:03.905330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:03.906046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:03.906651 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:03.907514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:03.908136 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:03.908753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:04.143237 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:04.155465 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-08e69f12-5ef7-45a0-9a97-2ee35938be54 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.268413 Oct 28 11:02:04.156048 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 70/131] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:03 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:04.278334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:04.279004 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:04.285933 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:04.286477 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:04.286914 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:04.287601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:04.288049 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:04.288715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:04.436876 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:04.445660 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-87231840-d453-4bc1-b440-28eeec052b03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.171191 Oct 28 11:02:04.446284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 62/132] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:04 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:04.879165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:04.879776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:04.885934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:04.886646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:04.887075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:04.888004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:04.888644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:04.889070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:05.040623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:05.052987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:05.055360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:05.056507 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-73d1cce6-fab7-43b2-a1d6-87bbcdc5f157 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.182086 Oct 28 11:02:05.057371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 71/133] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:04 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:05.178629 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:05.179225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:05.185540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:05.186167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:05.186605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:05.187292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:05.187735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:05.188324 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:05.340347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:05.352546 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ede362ff-f897-4cf7-98ee-fe881e413af4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.177642 Oct 28 11:02:05.355740 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 63/134] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:05 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:05.476472 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:05.477831 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:05.495181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:05.495181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:05.496458 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:05.497183 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:05.499120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:05.499561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:05.656670 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:05.665584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-410b2149-c816-4483-aec3-e7a2430975e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.201226 Oct 28 11:02:05.666842 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 72/135] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:05 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:06.084510 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:06.085297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:06.091189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:06.091662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:06.092089 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:06.092760 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:06.093411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:06.094136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:06.242673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:06.256610 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:06.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:06.261114 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-309d00b2-8d0e-4e0f-aee7-216c5d28f1f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.180384 Oct 28 11:02:06.261859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 64/136] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:06 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:06.380238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:06.381022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:06.388234 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:06.388665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:06.389092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:06.389772 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:06.390258 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:06.390685 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:06.528883 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:06.537621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f04e1643-019b-4b97-8120-8df85e9e0ef3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.161752 Oct 28 11:02:06.538539 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 73/137] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:06 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:06.696728 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:06.696728 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:06.700239 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:06.700727 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:06.701161 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:06.701847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:06.702337 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:06.702761 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:06.856964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:06.866717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-63eebc19-c30b-4725-85fe-dfe15795542c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.176684 Oct 28 11:02:06.878054 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 65/138] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:06 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:07.302448 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:07.303658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:07.311757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:07.312351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:07.313331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:07.314471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:07.314956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:07.316349 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:07.510142 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:07.518246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:07.519700 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:07.520782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-99ea9e2d-5519-4d3e-b2eb-c84fdd1bc32e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.222669 Oct 28 11:02:07.521528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 74/139] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:07 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:07.562151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:07.562771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:07.570689 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:07.571316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:07.571889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:07.572809 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:07.573416 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:07.573910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:07.712172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:07.720650 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c6aa2c62-9fd8-4463-9faf-de6ff8a62595 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.163206 Oct 28 11:02:07.721360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 66/140] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:07 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:07.890456 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:07.891059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:07.892860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:07.898321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:07.898789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:07.899213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:07.899892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:07.900337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:07.900748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:08.070404 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:08.078076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-31b4c897-ddbd-4b04-9273-8db0a52efd52 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.191350 Oct 28 11:02:08.078644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 75/141] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:07 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:08.544612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:08.545225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:08.551079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:08.551556 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:08.551981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:08.552664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:08.553129 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:08.553545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:08.710207 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:08.730814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:08.732673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:08.741236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4b06b1bb-593f-4963-bca1-dc43ae577920 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.198147 Oct 28 11:02:08.745257 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:08.746465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 67/142] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:08 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:08.749405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:08.762752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:08.764625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:08.766964 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:08.770419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:08.773460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:08.773914 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:09.103756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:09.104726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:09.119327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:09.121308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:09.122684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:09.125124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:09.127184 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:09.128134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:09.327758 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:09.338297 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-acff63a8-5c84-4991-a92b-dd4f70689688 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.597292 Oct 28 11:02:09.344730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 76/143] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:08 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:09.407563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:09.418360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-296fdef0-10b9-4bce-b7dc-6faef4828d2f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.321388 Oct 28 11:02:09.431106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 68/144] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:09 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:09.771519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:09.772398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:09.778737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:09.779370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:09.780106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:09.781088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:09.781600 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:09.782216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:09.984884 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:09.994918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:09.996032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:09.997371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cbdd6482-d420-46e8-9931-76c8ed48b580 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.230699 Oct 28 11:02:10.010396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 77/145] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:09 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:10.023644 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-8c24f1ab-7cff-4147-8ee3-b545b9301894 req-8571e5e4-66f7-4f8d-b539-047e303df184 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4ef9cf5-ad72-4399-a910-9308a243a57a", "tag": "06fb27d9-2d31-41b8-ad55-60bad7a8ec79"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:10.037634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-8c24f1ab-7cff-4147-8ee3-b545b9301894 req-8571e5e4-66f7-4f8d-b539-047e303df184 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:10.038154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-8c24f1ab-7cff-4147-8ee3-b545b9301894 req-8571e5e4-66f7-4f8d-b539-047e303df184 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:10.038582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-8c24f1ab-7cff-4147-8ee3-b545b9301894 req-8571e5e4-66f7-4f8d-b539-047e303df184 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:10.055308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-8c24f1ab-7cff-4147-8ee3-b545b9301894 req-8571e5e4-66f7-4f8d-b539-047e303df184 service nova] Creating event network-changed:06fb27d9-2d31-41b8-ad55-60bad7a8ec79 for instance e4ef9cf5-ad72-4399-a910-9308a243a57a on hv-cinder-zed-3 Oct 28 11:02:10.061732 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-8c24f1ab-7cff-4147-8ee3-b545b9301894 req-8571e5e4-66f7-4f8d-b539-047e303df184 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043010 Oct 28 11:02:10.062563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 69/146] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:02:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:02:10.367946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:10.368551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:10.374555 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:10.375032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:10.375455 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:10.376126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:10.376574 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:10.376988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:10.446069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:10.446836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:10.455327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:10.456479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:10.457304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:10.458317 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:10.458927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:10.459329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:10.536016 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:10.543848 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a6bb1ffa-4a6d-4c67-b1b2-0d34dbd3f6d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.180078 Oct 28 11:02:10.544639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 78/147] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:10 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:10.607884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:10.616577 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0cd4657d-cc81-4be4-a9c8-728b2817f552 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.174624 Oct 28 11:02:10.617177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 70/148] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:10 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:11.024371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:11.024967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:11.030931 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:11.031402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:11.031826 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:11.032493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:11.032934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:11.033355 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:11.156107 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:11.159480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:11.164846 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:11.166352 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:11.167428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b50c7fba-7b0b-44f6-8202-d23bff141098 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.147515 Oct 28 11:02:11.167980 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 79/149] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:11 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:11.566205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:11.566734 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:11.572526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:11.573002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:11.573432 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:11.574133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:11.574592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:11.575008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:11.641324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:11.642185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:11.648785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:11.649269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:11.649703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:11.650753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:11.651426 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:11.651614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:11.705545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:11.721599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-68846b48-38de-4bd3-aa17-6e638e49b1d9 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.159300 Oct 28 11:02:11.722217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 71/150] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:11 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:11.779325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:11.786913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ce16399d-5157-4b5c-9e00-025dce9596e1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.150214 Oct 28 11:02:11.788099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 80/151] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:11 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:12.191502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:12.192071 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:12.198378 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:12.198838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:12.199256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:12.199962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:12.200387 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:12.200795 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:12.219327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:12.330607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:12.339207 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:12.340633 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:12.341745 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e31c0834-5705-4267-8ff2-f6200b7503e3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.154180 Oct 28 11:02:12.342322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 72/152] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:12 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:12.745381 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:12.746035 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:12.751926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:12.752396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:12.752823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:12.753587 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:12.753927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:12.754361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:12.810128 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:12.810867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:12.816724 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:12.817203 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:12.817634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:12.818964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:12.819427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:12.819857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:12.884020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:12.891604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c6bb54e8-00b3-4cbe-a25b-3cee4707e67e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.149882 Oct 28 11:02:12.892100 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 81/153] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:12 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:12.943339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:12.950818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c6c629a4-257f-470d-a3d5-81acf17313ea tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.144621 Oct 28 11:02:12.951376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 73/154] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:12 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:13.367012 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:13.367615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:13.373492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:13.373937 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:13.374388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:13.375065 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:13.375508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:13.375921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:13.499677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:13.507656 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:13.509094 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:13.510220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ffa8aa5f-5ac6-4563-b74c-e67ae0c2fc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.147222 Oct 28 11:02:13.510781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 82/155] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:13 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:13.914807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:13.915342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:13.921372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:13.921851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:13.922307 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:13.922992 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:13.923440 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:13.923855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:13.972556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:13.973158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:13.979866 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:13.980452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:13.980865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:13.982125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:13.982244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:13.982645 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:14.129686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:14.137634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-de99c9b2-8113-4a50-819d-a9f0f9249807 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.168796 Oct 28 11:02:14.138453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 83/156] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:13 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:14.217762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:14.226298 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-14e8e998-8a3f-4902-8b65-bad9a56950be tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.315803 Oct 28 11:02:14.226862 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 74/157] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:13 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:14.535416 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:14.535926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:14.541830 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:14.542461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:14.542896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:14.543573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:14.544026 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:14.544446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:14.672945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:14.681161 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:14.682589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:14.683674 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5adb786c-04fe-4283-b557-20a4ca203f8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.152315 Oct 28 11:02:14.684220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 84/158] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:14 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:15.161159 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:15.163005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:15.169355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:15.170041 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:15.170728 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:15.171502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:15.172028 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:15.172671 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:15.249411 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:15.250781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:15.264237 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:15.264914 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:15.265504 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:15.266370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:15.266824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:15.267249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:15.393357 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:15.403009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5a792827-c299-4fb5-8f08-c756f3ce421f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.245836 Oct 28 11:02:15.418028 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 75/159] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:15 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:15.456496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:15.468276 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1ce66ff2-27f4-4d1f-a52f-ba647ca63917 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.223484 Oct 28 11:02:15.468843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 85/160] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:15 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:15.650971 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-ca06d9d9-f888-4e9f-afdd-f9e1c91619b6 req-354f29f9-ff2b-4d70-9668-14ba2d6a207e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd2715aa-33ac-4197-90f4-d2dc79a7f07e", "name": "network-vif-plugged", "status": "completed", "tag": "0da3d13b-05f9-481e-8125-00a60da3dc59"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:15.656270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=76 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:15.663309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ca06d9d9-f888-4e9f-afdd-f9e1c91619b6 req-354f29f9-ff2b-4d70-9668-14ba2d6a207e service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:15.663876 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ca06d9d9-f888-4e9f-afdd-f9e1c91619b6 req-354f29f9-ff2b-4d70-9668-14ba2d6a207e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:15.664303 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ca06d9d9-f888-4e9f-afdd-f9e1c91619b6 req-354f29f9-ff2b-4d70-9668-14ba2d6a207e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:15.676983 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-ca06d9d9-f888-4e9f-afdd-f9e1c91619b6 req-354f29f9-ff2b-4d70-9668-14ba2d6a207e service nova] Creating event network-vif-plugged:0da3d13b-05f9-481e-8125-00a60da3dc59 for instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e on hv-cinder-zed-3 Oct 28 11:02:15.682857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-ca06d9d9-f888-4e9f-afdd-f9e1c91619b6 req-354f29f9-ff2b-4d70-9668-14ba2d6a207e service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094136 Oct 28 11:02:15.683439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 76/161] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:02:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:02:15.709917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:15.710543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:15.716544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:15.717008 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:15.717433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:15.718140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:15.718908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:15.719326 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:15.861121 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:15.869220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:15.870732 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:15.871822 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3572c2fc-8068-4d22-b63a-75eb27b23a67 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.167003 Oct 28 11:02:15.872377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 86/162] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:15 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:16.430251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:16.431791 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:16.439497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:16.439901 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:16.444330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:16.445700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:16.446571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:16.447530 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:16.496737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:16.496737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:16.503759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:16.504248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:16.507355 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:16.508235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:16.509049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:16.509626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:16.599976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:16.610304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d376b680-b366-448f-8bb2-9500a896aad2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.185871 Oct 28 11:02:16.611681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 77/163] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:16 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:16.746361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:16.758453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7488f0a8-6938-4317-b369-73088c11d0a4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.269382 Oct 28 11:02:16.759690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 87/164] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:16 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:16.899674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:16.900841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:16.911813 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:16.913347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:16.913945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:16.914889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:16.915984 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:16.917258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:17.128586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:17.141428 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:17.143859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:17.145386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-afa88e64-398e-455f-a5af-0d50fab39ac1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.251162 Oct 28 11:02:17.146835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 78/165] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:16 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:17.633666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:17.634446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:17.645336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:17.646097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:17.647670 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:17.648531 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:17.648974 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:17.649393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:17.783411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:17.784025 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:17.789939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:17.790458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:17.790894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:17.791613 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:17.792069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:17.792494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:17.877814 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:17.889808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f71f9f4b-986a-447d-9157-dd48f54d6d7f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.260419 Oct 28 11:02:17.906817 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 88/166] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:17 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:17.988013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:17.998707 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-303f5062-1262-44e7-9616-86b7253cb119 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.219381 Oct 28 11:02:17.999682 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 79/167] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:17 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:18.172725 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:18.173791 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:18.180881 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:18.181367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:18.181795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:18.182722 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:18.183313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:18.183938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:18.361413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:18.372116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:18.374126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:18.375789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-475612a3-0654-4a22-9dfe-cdee5ceb01a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.208206 Oct 28 11:02:18.376500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 89/168] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:18 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:18.915934 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:18.917213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:18.924241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:18.924731 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:18.925313 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:18.926028 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:18.926678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:18.927215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:19.025163 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:19.026623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:19.051516 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:19.052388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:19.053669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:19.054479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:19.055124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:19.055538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:19.243426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:19.257351 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ad184d4d-4e8a-4a55-b8b6-06c3aa01df03 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.346620 Oct 28 11:02:19.267208 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 80/169] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:18 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:19.362640 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-53454245-ff8f-49e5-bcea-d504e87a6df1 req-d3645d3e-677e-4d11-99b2-93b951bf9531 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54cf39f2-cfd4-4f12-bddb-36666db9e1d5", "name": "network-vif-plugged", "status": "completed", "tag": "9cdeceb0-c9f5-42e0-9e0e-62d116b04c2f"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:19.388898 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-53454245-ff8f-49e5-bcea-d504e87a6df1 req-d3645d3e-677e-4d11-99b2-93b951bf9531 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:19.391359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-53454245-ff8f-49e5-bcea-d504e87a6df1 req-d3645d3e-677e-4d11-99b2-93b951bf9531 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:19.393093 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-53454245-ff8f-49e5-bcea-d504e87a6df1 req-d3645d3e-677e-4d11-99b2-93b951bf9531 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:19.430250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-53454245-ff8f-49e5-bcea-d504e87a6df1 req-d3645d3e-677e-4d11-99b2-93b951bf9531 service nova] Creating event network-vif-plugged:9cdeceb0-c9f5-42e0-9e0e-62d116b04c2f for instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 on hv-cinder-zed-3 Oct 28 11:02:19.441406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-53454245-ff8f-49e5-bcea-d504e87a6df1 req-d3645d3e-677e-4d11-99b2-93b951bf9531 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084513 Oct 28 11:02:19.442374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 81/170] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:02:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:02:19.453261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:19.458293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:19.462789 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:19.469924 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0fc94208-18d7-4f89-a86d-a497a63ffc49 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.450983 Oct 28 11:02:19.473708 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 90/171] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:19 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:19.478691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:19.479250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:19.479741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:19.480504 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:19.481015 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:19.481492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:19.705590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:19.716101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:19.717539 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:19.718664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-25628469-faef-496b-93bf-cab4dfa447f5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.273059 Oct 28 11:02:19.719525 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 82/172] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:19 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:20.281200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:20.281817 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:20.283627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=82 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:20.289433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:20.290046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:20.290421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:20.291423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:20.291987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:20.292573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:20.432565 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:20.442508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e8858053-8a98-43f2-b449-5cb7a476c912 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.163962 Oct 28 11:02:20.442508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 91/173] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:20 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:20.500815 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:20.501488 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:20.508885 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:20.509604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:20.510309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:20.511142 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:20.511592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:20.512008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:20.659260 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:20.667056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e15430ba-4d6a-4e81-b0cb-08beb393083e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.170329 Oct 28 11:02:20.667628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 83/174] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:20 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:20.745262 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:20.745867 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:20.751718 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:20.752188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:20.752798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:20.753479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:20.754116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:20.754823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:20.908465 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:20.916455 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:20.917965 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:20.919266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7a097f6c-0a61-4cab-91ba-789cac35f73f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.177917 Oct 28 11:02:20.919976 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 92/175] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:20 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:21.463975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:21.464920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:21.471060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:21.471759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:21.472428 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:21.473187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:21.473714 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:21.474807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:21.626172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:21.635833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3d5cd39c-359a-410e-9f0a-f37f8c9d6ac7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.175639 Oct 28 11:02:21.636654 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 84/176] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:21 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:21.690268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:21.691659 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:21.698677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:21.699159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:21.699997 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:21.700855 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:21.701457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:21.702051 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:21.846089 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:21.855834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d71be808-553d-46e1-8e8d-b0281b00a732 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.169439 Oct 28 11:02:21.857097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 93/177] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:21 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:21.944475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:21.945283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:21.951978 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:21.952693 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:21.953230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:21.954063 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:21.954612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:21.955261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:22.093847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:22.103828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:22.105701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:22.106932 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5329ccaf-366f-45e3-b031-234d886edc32 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.166449 Oct 28 11:02:22.107666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 85/178] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:21 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:22.660001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:22.660610 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:22.681904 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:22.683409 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:22.683986 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:22.684852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:22.685864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:22.692445 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:22.881401 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:22.882119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:22.893034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:22.898608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:22.901087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:22.903262 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:22.905430 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:22.906621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:22.950215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:22.965617 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4c71e2cf-e621-4d38-a9f6-2cb5806d8f67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.308531 Oct 28 11:02:22.966251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 94/179] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:22 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:23.134562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:23.135253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:23.146423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:23.146423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:23.146423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:23.146423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:23.146760 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:23.147446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:23.234782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:23.244435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6c7c881c-125b-444f-98b0-5ab1d3a6cd14 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1314 microversion: 2.1 time: 0.367994 Oct 28 11:02:23.245633 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 86/180] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:22 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1314 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:23.448210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:23.457597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:23.459516 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:23.461314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ad52fd56-1a70-432c-909e-80f1e2ddfa73 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.331365 Oct 28 11:02:23.462804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 95/181] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:23 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:23.607494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-8d18c5de-995d-4aa6-83a7-568ee890dfbb req-4c47d8e8-f1e2-4698-a85d-3ff86b890c50 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "643427ec-cf7f-4058-a731-b96412826c2e", "name": "network-vif-plugged", "status": "completed", "tag": "c012cb5d-de75-4d0c-8ffe-7a84c1ef9d66"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:23.628794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-8d18c5de-995d-4aa6-83a7-568ee890dfbb req-4c47d8e8-f1e2-4698-a85d-3ff86b890c50 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:23.629507 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-8d18c5de-995d-4aa6-83a7-568ee890dfbb req-4c47d8e8-f1e2-4698-a85d-3ff86b890c50 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:23.629929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-8d18c5de-995d-4aa6-83a7-568ee890dfbb req-4c47d8e8-f1e2-4698-a85d-3ff86b890c50 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:23.644249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-8d18c5de-995d-4aa6-83a7-568ee890dfbb req-4c47d8e8-f1e2-4698-a85d-3ff86b890c50 service nova] Creating event network-vif-plugged:c012cb5d-de75-4d0c-8ffe-7a84c1ef9d66 for instance 643427ec-cf7f-4058-a731-b96412826c2e on hv-cinder-zed-3 Oct 28 11:02:23.653245 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-8d18c5de-995d-4aa6-83a7-568ee890dfbb req-4c47d8e8-f1e2-4698-a85d-3ff86b890c50 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053989 Oct 28 11:02:23.654251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 87/182] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:02:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:02:23.993623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:23.994864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:24.008995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:24.009496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:24.009956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:24.013843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:24.014347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:24.017753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:24.073219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:24.276681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:24.279073 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:24.287490 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:24.288280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:24.289036 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:24.290291 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:24.290950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:24.291572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:24.345116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:24.355413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-25e110a2-da66-468f-9214-aeb81c267ecf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.366260 Oct 28 11:02:24.356000 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 96/183] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:23 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:24.492500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:24.493526 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:24.500786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:24.502451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:24.504962 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:24.507192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:24.508545 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:24.509287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:24.627820 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:24.639040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a718944d-e811-4893-83f9-d886abb75988 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1501 microversion: 2.1 time: 0.367719 Oct 28 11:02:24.639504 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 88/184] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:24 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1501 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:24.727254 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:24.744299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:24.752855 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:24.761864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c2ff4201-ed8e-4601-9e09-f01b57673a4d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.274336 Oct 28 11:02:24.762599 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 97/185] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:24 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:02:25.379256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:25.379952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:25.390500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:25.391845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:25.394057 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:25.394891 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:25.398447 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:25.400273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:25.650033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:25.658097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6e022ce6-79e1-46ed-b972-556dbae2ca4b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1322 microversion: 2.1 time: 0.282561 Oct 28 11:02:25.659623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 89/186] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:25 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1322 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:25.684946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-5d21237a-3a76-4a90-9542-8eca71b7e0c3 req-e9cc8bd1-f1f1-42f3-a20f-e0d4bdaed0aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ef9cf5-ad72-4399-a910-9308a243a57a", "name": "network-vif-plugged", "status": "completed", "tag": "06fb27d9-2d31-41b8-ad55-60bad7a8ec79"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:25.703431 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-5d21237a-3a76-4a90-9542-8eca71b7e0c3 req-e9cc8bd1-f1f1-42f3-a20f-e0d4bdaed0aa service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:25.704559 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-5d21237a-3a76-4a90-9542-8eca71b7e0c3 req-e9cc8bd1-f1f1-42f3-a20f-e0d4bdaed0aa service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:25.706829 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-5d21237a-3a76-4a90-9542-8eca71b7e0c3 req-e9cc8bd1-f1f1-42f3-a20f-e0d4bdaed0aa service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:25.723279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-5d21237a-3a76-4a90-9542-8eca71b7e0c3 req-e9cc8bd1-f1f1-42f3-a20f-e0d4bdaed0aa service nova] Creating event network-vif-plugged:06fb27d9-2d31-41b8-ad55-60bad7a8ec79 for instance e4ef9cf5-ad72-4399-a910-9308a243a57a on hv-cinder-zed-3 Oct 28 11:02:25.732869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-5d21237a-3a76-4a90-9542-8eca71b7e0c3 req-e9cc8bd1-f1f1-42f3-a20f-e0d4bdaed0aa service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052546 Oct 28 11:02:25.738867 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 98/187] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:02:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:02:25.786236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:25.786989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:25.792868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:25.793342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:25.793762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:25.794470 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:25.794915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:25.795331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:25.931581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:25.941995 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:25.944221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:25.945311 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a3caf7d6-ce3b-45be-aeb4-d4d025b8ed89 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.163084 Oct 28 11:02:25.945861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 90/188] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:25 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:26.682921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:26.683612 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:26.689964 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:26.690535 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:26.691503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:26.692535 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:26.693434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:26.693852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:26.853594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:26.861692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1e99e1a7-5149-4a50-a369-643c6eba679a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1516 microversion: 2.1 time: 0.183051 Oct 28 11:02:26.870130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 99/189] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:26 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1516 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:26.887918 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:26.890866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:26.897615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:26.898209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:26.899203 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:26.900088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:26.900540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:26.900960 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:26.931244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:26.968686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lazy-loading 'flavor' on Instance uuid 643427ec-cf7f-4058-a731-b96412826c2e {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:02:26.978212 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:26.978212 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:26.978807 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:26.979392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:26.979926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:26.980804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:26.981475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:26.982012 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:27.123814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dbf19257-c69c-4bf4-99f1-c8c655b824c1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e/action" status: 200 len: 32 microversion: 2.1 time: 0.241328 Oct 28 11:02:27.124426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 91/190] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:02:26 2022] POST /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e/action => generated 32 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 28 11:02:27.135458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:27.136333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:27.142530 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:27.143017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:27.143454 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:27.144144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:27.144594 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:27.145018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:27.146532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:27.156520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:27.158554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:27.159663 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b54b20bc-a901-4adb-93b9-95d24e2bbb02 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.193781 Oct 28 11:02:27.160222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 100/191] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:26 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:27.281045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:27.289225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-755d99c6-dab9-41ea-8c54-a8cb46d81979 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.158983 Oct 28 11:02:27.289780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 92/192] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:27 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:28.184687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:28.185281 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:28.191293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:28.192046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:28.192473 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:28.193197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:28.193639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:28.194087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:28.313480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:28.314100 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:28.321170 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:28.321687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:28.322139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:28.323015 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:28.323457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:28.323873 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:28.327031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:28.335623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:28.337343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:28.338756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-212e5284-cccc-436a-80e9-7650a09cc0ca tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.157922 Oct 28 11:02:28.339553 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 101/193] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:28 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:28.464286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:28.474115 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ac9da01b-f1f4-4e94-ad82-f8708869bf56 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.164836 Oct 28 11:02:28.491940 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 93/194] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:28 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:29.364994 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:29.365677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:29.371552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:29.372031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:29.372492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:29.373208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:29.373662 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:29.374123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:29.498358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:29.498966 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:29.506753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:29.507388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:29.507951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:29.508709 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:29.509154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:29.509446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:29.509605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:29.518274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:29.520346 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:29.522517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2da45c82-a228-49d6-a351-0826bf6f1599 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.161517 Oct 28 11:02:29.523070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 102/195] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:29 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:29.644072 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:29.652674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-90da0638-859e-4609-a76c-a86e9e243014 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.158083 Oct 28 11:02:29.653257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 94/196] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:29 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:30.547482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:30.548099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:30.554348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:30.554835 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:30.555266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:30.555967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:30.556420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:30.556843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:30.676652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:30.677284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:30.683586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:30.684198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:30.684624 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:30.685307 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:30.685785 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:30.686220 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:30.723517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:30.735254 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:30.736860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:30.738154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f8397caa-42a5-41ee-af75-1f630c7e73d6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1406 microversion: 2.20 time: 0.194752 Oct 28 11:02:30.738707 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 103/197] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:30 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1406 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:30.825032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:30.833156 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-968394dd-283b-48d4-9134-101f3a624ef6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.160778 Oct 28 11:02:30.834386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 95/198] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:30 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:31.763820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:31.764433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:31.770320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:31.770785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:31.771209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:31.771876 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:31.772322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:31.772731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:31.856493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:31.857202 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:31.863424 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:31.863904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:31.864335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:31.865854 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:31.866327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:31.866750 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:31.902980 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:31.912894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:31.914391 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:31.915605 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0e2e43e0-c942-47db-8238-d8ad78bf3d1d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1598 microversion: 2.20 time: 0.155828 Oct 28 11:02:31.916163 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 104/199] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:02:31 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1598 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:02:31.942128 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:31.942624 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:31.948824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:31.949296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:31.949718 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:31.950494 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:31.950939 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:31.951349 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:32.002720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:32.011322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c262f1fd-3581-4d81-81b7-55b63943711e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.158581 Oct 28 11:02:32.011888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 96/200] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:31 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:32.065108 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9a4c413e-54af-4523-89fc-1d50a84c24ab"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:32.069097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:32.075935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:32.076480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:32.076944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:32.077648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3f765069-4d8d-4f7c-a376-58c1c8f35788 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.139608 Oct 28 11:02:32.078363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 105/201] 10.222.0.46 () {60 vars in 1316 bytes} [Fri Oct 28 11:02:31 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-interface => generated 285 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 11:02:32.078632 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:32.079234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:32.079662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:33.035805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:33.036452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:33.042400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:33.042885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:33.043321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:33.044215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:33.044671 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:33.045101 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:33.246217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:33.254128 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-76c2c477-9b57-4745-9dc2-40e3626a6c70 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.222359 Oct 28 11:02:33.254917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 106/202] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:33 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:33.387701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-88afeef5-df22-4524-b63e-e11977a69ede req-f3e16426-6c36-4daa-a2d8-2baaa5217c1f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4ef9cf5-ad72-4399-a910-9308a243a57a", "tag": "06fb27d9-2d31-41b8-ad55-60bad7a8ec79"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:02:33.397127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-88afeef5-df22-4524-b63e-e11977a69ede req-f3e16426-6c36-4daa-a2d8-2baaa5217c1f service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:33.397594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-88afeef5-df22-4524-b63e-e11977a69ede req-f3e16426-6c36-4daa-a2d8-2baaa5217c1f service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:33.398036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-88afeef5-df22-4524-b63e-e11977a69ede req-f3e16426-6c36-4daa-a2d8-2baaa5217c1f service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:33.408622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-88afeef5-df22-4524-b63e-e11977a69ede req-f3e16426-6c36-4daa-a2d8-2baaa5217c1f service nova] Creating event network-changed:06fb27d9-2d31-41b8-ad55-60bad7a8ec79 for instance e4ef9cf5-ad72-4399-a910-9308a243a57a on hv-cinder-zed-3 Oct 28 11:02:33.414129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-88afeef5-df22-4524-b63e-e11977a69ede req-f3e16426-6c36-4daa-a2d8-2baaa5217c1f service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030057 Oct 28 11:02:33.414710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 107/203] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:02:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:02:34.277757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:34.278499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:34.284891 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:34.285370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:34.285798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:34.287229 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:34.287690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:34.288110 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:34.609681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:34.617382 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5b0a515b-067f-45f9-a8aa-58d7159f8bf4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.343785 Oct 28 11:02:34.617870 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 108/204] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:34 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:35.639789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:35.640425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:35.646278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:35.646757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:35.647188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:35.647903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:35.648355 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:35.648771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:35.779635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:35.788149 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2ab0e7b4-829f-4012-b68d-56dca58c0d9e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.152135 Oct 28 11:02:35.788785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 109/205] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:35 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:36.348298 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c6251a0a-1327-4f02-8314-610ada823921 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "POST /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.287525 Oct 28 11:02:36.352331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 97/206] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:02:32 2022] POST /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e/os-volume_attachments => generated 194 bytes in 4290 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:02:36.812441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:36.813452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:36.820177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:36.820805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:36.821339 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:36.822134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:36.822786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:36.823213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:36.957067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:36.965502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f3628f41-8b56-46dd-9f30-4cb61d8e3ac5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.157040 Oct 28 11:02:36.981191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 110/207] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:36 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:37.989452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:37.990180 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:37.998645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:37.999127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:37.999560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:38.000254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:38.000705 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:38.001165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:38.133890 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:38.141581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-49fd3977-7e06-4653-8a4b-28356588082d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.156401 Oct 28 11:02:38.142176 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 98/208] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:37 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:39.165001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:39.165608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:39.172023 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:39.172494 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:39.172925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:39.173608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:39.174087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:39.174506 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:39.303187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:39.310722 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-40a97689-ec72-4d10-afa9-e094bdbd9521 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.150234 Oct 28 11:02:39.311279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 111/209] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:39 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:40.334238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:40.334845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:40.341358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:40.341841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:40.342316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:40.342999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:40.343449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:40.343872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:40.473583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:40.481869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7beebea6-7afc-4424-b670-ff0f2914b92d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.151966 Oct 28 11:02:40.482459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 99/210] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:40 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:41.504989 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:41.505527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:41.511424 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:41.511908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:41.512341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:41.513020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:41.513471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:41.513892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:41.647397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:41.654890 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c9a8784b-e0a7-4b6c-96c4-2dcb2ef467af tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.153824 Oct 28 11:02:41.655446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 112/211] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:41 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:42.678548 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:42.679057 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:42.685364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:42.685838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:42.686307 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:42.686998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:42.687458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:42.687887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:42.839989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:42.851498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0a060fc0-47b2-4be4-8628-88c0180c0660 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.176813 Oct 28 11:02:42.852171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 100/212] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:42 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:43.874845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:43.875447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:43.881391 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:43.881861 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:43.882880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:43.883554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:43.884002 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:43.884418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:44.013679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:44.021208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d736dd37-6c2c-4aeb-95ad-8b18d9b80617 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.150178 Oct 28 11:02:44.021762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 113/213] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:43 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:45.043366 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:45.043968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:45.050240 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:45.050722 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:45.051152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:45.051829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:45.052309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:45.052726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:45.179873 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:45.187508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c85098cb-535c-479d-8284-40236d9dd3d0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.147907 Oct 28 11:02:45.188075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 101/214] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:45 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:46.210036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:46.211153 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:46.217116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:46.217580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:46.218021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:46.218689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:46.219131 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:46.219542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:46.342747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:46.346325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:46.351661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9621bc9a-5868-40b3-a9ed-9d41764ab600 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.144791 Oct 28 11:02:46.352235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 114/215] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:46 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:47.374867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:47.375469 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:47.378242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=52 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:47.384179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:47.384650 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:47.385075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:47.385747 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:47.386217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:47.386638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:47.517967 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:47.526636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ae00d92c-e78b-47e9-b982-7d4ceabbb49f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.155589 Oct 28 11:02:47.527218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 102/216] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:47 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:48.548440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:48.549077 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:48.555804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:48.556281 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:48.556708 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:48.557385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:48.557834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:48.558293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:48.684403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:48.691975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-df959ff2-f309-4fb1-9905-c5b4894041c2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.147344 Oct 28 11:02:48.692543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 115/217] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:48 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:49.715856 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:49.716498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:49.723133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:49.723609 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:49.724037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:49.724714 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:49.725167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:49.725588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:49.854715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:49.862559 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d82d57e4-53e2-4594-8b72-6fa7ebe226be tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.151323 Oct 28 11:02:49.863120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 103/218] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:49 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:50.884754 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:50.885358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:50.887304 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=50 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:50.892609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:50.893084 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:50.893512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:50.894212 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:50.894661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:50.895076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:51.019113 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:51.028064 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e7957fb3-94e6-4385-a3c0-2dbe48f45ea1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.147185 Oct 28 11:02:51.028626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 116/219] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:50 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:52.051034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:52.051644 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:52.058635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:52.059116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:52.059552 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:52.060233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:52.060695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:52.061115 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:52.079162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:02:52.195467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:52.203084 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9750f316-dbe8-4217-9d7b-1a34abefae30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.155895 Oct 28 11:02:52.203844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 104/220] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:52 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:53.227125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:53.227735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:53.233581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:53.234360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:53.234798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:53.235483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:53.235938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:53.236361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:53.360804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:53.368922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0d9931b5-3663-4f36-aa3f-c7ce0f4f635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.145627 Oct 28 11:02:53.369475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 117/221] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:53 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:54.391231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:54.391841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:54.398702 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:54.399182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:54.399616 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:54.400299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:54.400752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:54.401175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:54.525662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:54.533191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-853a8e15-8e8e-44ce-8095-abcab4ef8285 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.145779 Oct 28 11:02:54.533757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 105/222] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:54 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:02:55.555231 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:55.555838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:55.561731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:55.562259 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:55.562695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:55.563374 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:55.563825 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:55.564243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:55.692669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:55.701511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b3746fac-4192-41b1-bead-3ec3f8d6cb67 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.150027 Oct 28 11:02:55.702208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 118/223] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:55 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:56.725267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:56.726687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:56.732944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:56.733418 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:56.734145 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:56.734828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:56.735278 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:56.735696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:56.872752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:56.885647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-eab2c267-f13c-4762-a8d9-f80801299dcb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.164213 Oct 28 11:02:56.886258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 106/224] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:56 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:57.911304 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:57.911913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:57.919400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:57.919875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:57.920299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:57.920978 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:57.921418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:57.922062 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:58.128561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:58.137420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-77390499-af21-41d8-879a-fbd185b5bf19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.229954 Oct 28 11:02:58.138324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 119/225] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:57 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:02:59.163477 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:02:59.164241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:02:59.175343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:59.175706 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:59.176139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:59.177164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:02:59.177576 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:02:59.178020 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:02:59.365298 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:02:59.374102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-acfb892c-c181-49c8-8bbd-827865a60d25 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.214967 Oct 28 11:02:59.374638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 107/226] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:02:59 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:00.399821 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:00.401649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:00.408433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:00.408987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:00.409495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:00.410305 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:00.410819 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:00.411430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:00.581407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:00.589628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-86baa915-66e5-41f2-9c27-a3711de6db46 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.195189 Oct 28 11:03:00.590299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 120/227] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:00 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:01.612978 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:01.614305 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:01.621837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:01.622136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:01.622738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:01.623427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:01.623870 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:01.624447 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:02.001086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:02.009811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a02ef18f-2b35-411f-bf8e-2538fe41925f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.400851 Oct 28 11:03:02.010451 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 108/228] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:01 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:03.038666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:03.039785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:03.065273 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:03.065772 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:03.071100 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:03.071860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:03.072337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:03.072769 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:03.388148 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:03.399027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-33ea9034-8607-4739-ae64-5c825e7fdd71 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.368784 Oct 28 11:03:03.400044 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 121/229] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:03 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:04.423135 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:04.424319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:04.431550 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:04.432406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:04.433083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:04.434233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:04.434934 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:04.435590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:04.581872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:04.590835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-52b13c36-088b-4020-b36e-5b0f5dc0e6b3 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.172146 Oct 28 11:03:04.591417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 109/230] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:04 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:05.615088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:05.615797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:05.635048 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:05.635543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:05.636010 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:05.636714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:05.637201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:05.637628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:05.847037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:05.855477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d8e03bcf-0d59-4765-b181-981e0a9c808f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.244539 Oct 28 11:03:05.856040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 122/231] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:05 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:06.882294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:06.883042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:06.890264 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:06.890846 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:06.891379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:06.892294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:06.892811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:06.893638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:07.045159 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:07.062589 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b91477b6-1bfb-4eb5-8f80-09d13ea05b61 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.181884 Oct 28 11:03:07.064987 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 110/232] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:06 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:08.092088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:08.093709 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:08.105796 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:08.107665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:08.110985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:08.112254 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:08.114731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:08.140061 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.027s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:08.538686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:08.551426 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b9020f04-ceb2-4ffd-81e4-7ea6937b4833 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.467680 Oct 28 11:03:08.557122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 123/233] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:08 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:09.579918 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:09.580538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:09.591825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:09.598732 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:09.599504 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:09.601144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:09.601610 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:09.602045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:09.695528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:09.696050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:09.703551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:09.705733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:09.706318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:09.707141 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:09.707790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:09.708331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:09.808627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:03:10.103249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:10.115360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-eba1d101-72ca-4f91-82c5-b66dc7b0480a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.539857 Oct 28 11:03:10.117820 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 111/234] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:09 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:10.122707 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6c237c7b-25b1-41a1-be54-42748c769251 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "DELETE /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 204 len: 0 microversion: 2.1 time: 0.431270 Oct 28 11:03:10.123561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 124/235] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:03:09 2022] DELETE /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:03:10.147134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:10.148125 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:10.156762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:10.158263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:10.159683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:10.160778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:10.165533 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:10.168169 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:10.509699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:10.521035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ebf545aa-b985-45c3-b1c0-8e19cbd2669e tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.383449 Oct 28 11:03:10.521623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 112/236] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:10 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:11.144039 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:11.144724 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:11.160601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:11.161086 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:11.161696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:11.162628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:11.163995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:11.164563 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:11.426953 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:11.439907 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b01a91a1-1d30-4d13-a05a-1f6061948371 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.300223 Oct 28 11:03:11.441578 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 125/237] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:11 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:11.548985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:11.549604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:11.556643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:11.557130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:11.557681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:11.558290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:11.558749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:11.559180 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:11.718917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:11.730283 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c6c2405d-1d92-4c8e-9ae0-d4aa5e7427d4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.185856 Oct 28 11:03:11.731200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 126/238] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:11 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:03:11.738367 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-55fed501-ebd5-4a25-8117-41c806c3ecc1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Action: 'create', calling method: .version_select at 0x7f42fa985040>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1118103513", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL2ioYR8e0MV2lT9SnJPhA0tLPynWH47BBVnGskWYNxN3rTzMHtiRX6nt0zc09V/eg0LNBKlTqRecZ0KfXkpLYUU557fblOR8DPgvZwcWYN8MiEmGdR6KxxbRkwRheOYjQ=="}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:11.762421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-55fed501-ebd5-4a25-8117-41c806c3ecc1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:03:11.767008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-55fed501-ebd5-4a25-8117-41c806c3ecc1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for user 6b926f7161c0487a8c2b04ada3b374e3 and project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:03:11.812058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-55fed501-ebd5-4a25-8117-41c806c3ecc1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.384498 Oct 28 11:03:11.812663 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 113/239] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:03:11 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:03:12.464241 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:12.465546 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:12.472615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:12.473234 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:12.473762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:12.474870 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:12.475813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:12.476461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:12.669372 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:12.679730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-bdbb04da-9638-4046-9ed4-52e5d5b0d57f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.219274 Oct 28 11:03:12.680388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 127/240] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:12 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:12.758563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:12.760824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:12.769060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:12.769937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:12.770507 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:12.771266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:12.772004 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:12.772691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:12.961177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:12.970259 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ace1efd7-5ddf-478f-b0b3-16fd7fffe3bf tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.217959 Oct 28 11:03:12.970835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 114/241] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:12 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:13.703783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:13.704515 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:13.711162 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:13.711639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:13.712077 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:13.712758 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:13.713211 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:13.713630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:13.861743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:13.870192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-10243342-5b95-47b6-8c2f-cc61faf0a45f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.170142 Oct 28 11:03:13.870771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 128/242] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:13 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:13.997199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:13.997885 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:14.008343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:14.009181 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:14.010245 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:14.011618 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:14.012232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:14.012803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:14.212649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:14.221391 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6191828e-68d2-41f1-b133-18b455dc583b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.228078 Oct 28 11:03:14.222191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 115/243] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:13 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:14.894915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:14.895613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:14.905631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:14.906478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:14.907127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:14.908403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:14.908912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:14.909461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:15.075469 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:15.079519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:15.086307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-aae547fc-3ba7-4afc-833a-b0933090727e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.195498 Oct 28 11:03:15.087954 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 129/244] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:14 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:15.258540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:15.259932 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:15.293845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:15.294372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:15.294838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:15.295492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:15.297429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:15.297941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:15.539880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:15.550380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ca38487f-ccae-4fa5-a9e6-7b777ee2aa46 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.303832 Oct 28 11:03:15.551209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 116/245] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:15 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:16.117153 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:16.117910 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:16.130450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:16.131238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:16.131888 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:16.132783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:16.138838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:16.139592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:16.387028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:16.394647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6adcd00d-929b-41f5-bcbe-8076db99a690 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.284466 Oct 28 11:03:16.395219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 130/246] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:16 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:16.576511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:16.577197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:16.585253 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:16.585834 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:16.586519 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:16.587320 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:16.587844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:16.588338 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:16.718331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:16.726785 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9a2e8db1-1cbf-489f-a8e9-25726673fee3 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.154437 Oct 28 11:03:16.727351 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 117/247] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:16 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:03:17.419919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:17.420524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:17.427237 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:17.427725 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:17.428192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:17.428919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:17.429370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:17.429820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:17.566068 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:17.574501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6d2a4e70-ab46-41ab-bf96-8a1a0e448f04 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.158484 Oct 28 11:03:17.575070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 131/248] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:17 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:17.750438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:17.751177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:17.753473 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:17.759895 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:17.760323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:17.760764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:17.761453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:17.761907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:17.762358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:17.907265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:17.920560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-baad995f-a47f-4626-9e10-4facd9c8d27b tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.173739 Oct 28 11:03:17.934449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 118/249] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:17 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:18.598170 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:18.599569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:18.608723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:18.610856 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:18.611290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:18.612017 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:18.612459 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:18.612963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:18.853430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:18.867392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d4e4510e-aa93-4830-b5d1-09226fa68d65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.273045 Oct 28 11:03:18.874729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-e413c28d-1e63-4557-ae44-f40a5850d5f1 req-a915d8e7-7079-4702-988a-6807198f57d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd2715aa-33ac-4197-90f4-d2dc79a7f07e", "name": "network-vif-unplugged", "status": "completed", "tag": "0da3d13b-05f9-481e-8125-00a60da3dc59"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:18.883618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 132/250] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:18 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:18.896568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e413c28d-1e63-4557-ae44-f40a5850d5f1 req-a915d8e7-7079-4702-988a-6807198f57d6 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:18.897663 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e413c28d-1e63-4557-ae44-f40a5850d5f1 req-a915d8e7-7079-4702-988a-6807198f57d6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:18.898122 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e413c28d-1e63-4557-ae44-f40a5850d5f1 req-a915d8e7-7079-4702-988a-6807198f57d6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:18.915608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-e413c28d-1e63-4557-ae44-f40a5850d5f1 req-a915d8e7-7079-4702-988a-6807198f57d6 service nova] Creating event network-vif-unplugged:0da3d13b-05f9-481e-8125-00a60da3dc59 for instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e on hv-cinder-zed-3 Oct 28 11:03:18.921127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-e413c28d-1e63-4557-ae44-f40a5850d5f1 req-a915d8e7-7079-4702-988a-6807198f57d6 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.050158 Oct 28 11:03:18.921665 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 119/251] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:03:18 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:03:18.951478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:18.952740 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:18.961942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:18.962821 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:18.963316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:18.964733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:18.965203 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:18.965731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:19.156704 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:19.165339 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-475e6329-753d-42fb-8da0-a5ed14fbfba0 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.221820 Oct 28 11:03:19.166161 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 133/252] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:18 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:19.894920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:19.895541 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:19.901439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:19.901904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:19.902363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:19.903027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:19.903470 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:19.903883 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:20.034271 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:20.042474 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-cdd073b3-f344-4d26-83ff-7cb33d67bb6e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.152379 Oct 28 11:03:20.043035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 120/253] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:19 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:20.190509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:20.191117 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:20.197019 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:20.197500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:20.197933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:20.198635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:20.199087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:20.199509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:20.330784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:20.339496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fe80757c-6bcd-4b1b-a8e1-2370a411bffe tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.152830 Oct 28 11:03:20.340059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 134/254] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:20 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:21.065076 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:21.065686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:21.072679 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:21.073185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:21.073614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:21.074335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:21.074783 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:21.075200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:21.201262 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:21.209265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5d1550c5-0516-448d-a7c7-6fb5531a5128 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.147949 Oct 28 11:03:21.209841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 121/255] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:21 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:21.362711 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:21.363322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:21.365183 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:21.370453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:21.370931 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:21.371365 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:21.372040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:21.372507 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:21.372908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:21.502695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:21.511413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-02541b48-d6ee-4a3d-afa5-4c54d7660da4 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.152788 Oct 28 11:03:21.511981 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 135/256] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:21 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:22.232130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:22.232765 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:22.234988 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=4 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:22.240463 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:22.241442 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:22.241877 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:22.242580 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:22.243034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:22.243459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:22.245962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=95 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:22.380907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:22.389553 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ed655e0a-6b5d-440e-b1a3-883b9bc5a67d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.161276 Oct 28 11:03:22.390137 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 122/257] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:22 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:22.534182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:22.534854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:22.541353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:22.541840 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:22.542299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:22.542991 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:22.543441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:22.543865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:22.805548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:22.814864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ff3d4786-ae0b-4eba-8633-1c225e164700 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1553 microversion: 2.1 time: 0.284737 Oct 28 11:03:22.815472 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 136/258] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:22 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1553 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:23.415681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:23.416784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:23.418928 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-538db411-91ce-4fbe-93a9-94032cea9909 req-7c7c3fe6-b376-4544-bec5-c9d0d5cbf204 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd2715aa-33ac-4197-90f4-d2dc79a7f07e", "name": "network-vif-deleted", "tag": "0da3d13b-05f9-481e-8125-00a60da3dc59"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:23.424685 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:23.425675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:23.426675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:23.427601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:23.428066 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:23.428518 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:23.432269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-538db411-91ce-4fbe-93a9-94032cea9909 req-7c7c3fe6-b376-4544-bec5-c9d0d5cbf204 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:23.432749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-538db411-91ce-4fbe-93a9-94032cea9909 req-7c7c3fe6-b376-4544-bec5-c9d0d5cbf204 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:23.433176 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-538db411-91ce-4fbe-93a9-94032cea9909 req-7c7c3fe6-b376-4544-bec5-c9d0d5cbf204 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:23.445406 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-538db411-91ce-4fbe-93a9-94032cea9909 req-7c7c3fe6-b376-4544-bec5-c9d0d5cbf204 service nova] Creating event network-vif-deleted:0da3d13b-05f9-481e-8125-00a60da3dc59 for instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e on hv-cinder-zed-3 Oct 28 11:03:23.450790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-538db411-91ce-4fbe-93a9-94032cea9909 req-7c7c3fe6-b376-4544-bec5-c9d0d5cbf204 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035716 Oct 28 11:03:23.451397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 137/259] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:03:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:03:23.561950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:23.570253 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-bfd0f762-24fa-47ac-9ecd-419685a9ff90 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1522 microversion: 2.1 time: 0.160412 Oct 28 11:03:23.570812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 123/260] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:23 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:23.840286 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:23.840987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:23.847218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:23.847698 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:23.848130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:23.848855 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:23.849428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:23.849897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:23.915041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:23.923613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a1ecb4c5-4ed4-4cb5-b984-2ce5cdd37085 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1345 microversion: 2.1 time: 0.088425 Oct 28 11:03:23.924218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 138/261] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:23 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1345 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:24.595584 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:24.596192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:24.603163 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:24.603668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:24.604102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:24.604781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:24.605234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:24.605657 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:24.736429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:24.744058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-aa453f07-af26-41cc-9437-bd5442472429 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1502 microversion: 2.1 time: 0.152402 Oct 28 11:03:24.744621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 124/262] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:24 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1502 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:24.765899 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:24.766628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:24.772924 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:24.773392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:24.773821 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:24.774517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:24.774961 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:24.775375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:24.907499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:24.915409 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-bea7f42c-4a72-4825-9193-574b86d35354 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1516 microversion: 2.1 time: 0.153707 Oct 28 11:03:24.915972 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 139/263] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:24 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1516 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:24.947720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:24.948332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:24.954090 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:24.954576 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:24.955013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:24.955694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:24.956148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:24.956627 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:25.020367 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:25.029210 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-64485103-9994-4d44-ae93-f39b96878706 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1345 microversion: 2.1 time: 0.085992 Oct 28 11:03:25.029817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 125/264] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:24 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1345 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:26.058201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:26.060360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:26.068420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:26.069913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:26.070439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:26.071244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:26.072512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:26.072887 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:26.141868 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:26.151413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6693b180-d832-4be5-bf17-ef8f91ee8d69 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1345 microversion: 2.1 time: 0.101750 Oct 28 11:03:26.152226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 140/265] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:26 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1345 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:27.176098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:27.176894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:27.194974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:27.195454 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:27.196071 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:27.196776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:27.197224 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:27.197656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:27.274167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1903d1aaa3444181ac7b63234f4e8970', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '115fe2408cba49bb8b84af24fc8ce604', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:27.284300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ec07066c-e30f-48f9-aa73-a921d63828d6 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 200 len: 1345 microversion: 2.1 time: 0.113256 Oct 28 11:03:27.284860 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 126/266] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:27 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 1345 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:28.307861 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:28.308506 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:28.315466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.315985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.316761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:28.317567 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.318060 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.318854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:28.334295 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] HTTP exception thrown: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Request method failure captured: Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e HTTP/1.1 Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7aCeU30XGDIQxzF9h2t3eGZjB9CwS73-CeRmCBaa9AIZCfG4Gqlgx7HCtGRaxbTlBIdvJLIbbv_Y58TEWlh21c7HcQHmRfOXEiqi54OX2ueuEFc1mmWJb-rRfl65htdwFRXohPXbxq_fVL3_t4vMmZ4fkZdj1Kbc50PDnN66Xkr-DI Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:03:28.338935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 1903d1aaa3444181ac7b63234f4e8970 Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.340310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'bd2715aa-33ac-4197-90f4-d2dc79a7f07e'} Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:03:28.341079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.343074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.344071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Returning 404 to user: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:03:28.344071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a8ebe6de-dede-4d38-b198-dbc05f3ff15f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 404 len: 111 microversion: 2.1 time: 0.036971 Oct 28 11:03:28.344549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 141/267] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:28 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:03:28.499144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:28.501247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:28.508144 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:28.510446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:28.520554 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.521036 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.524307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.524791 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.525536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:28.526282 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.526738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.527153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:28.530657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:28.532018 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.533753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.534417 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:28.541054 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] HTTP exception thrown: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Request method failure captured: Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e/os-volume_attachments HTTP/1.1 Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aCeU30XGDIQxzF9h2t3eGZjB9CwS73-CeRmCBaa9AIZCfG4Gqlgx7HCtGRaxbTlBIdvJLIbbv_Y58TEWlh21c7HcQHmRfOXEiqi54OX2ueuEFc1mmWJb-rRfl65htdwFRXohPXbxq_fVL3_t4vMmZ4fkZdj1Kbc50PDnN66Xkr-DI Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:03:28.546327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 1903d1aaa3444181ac7b63234f4e8970 Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.547743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'server_id': 'bd2715aa-33ac-4197-90f4-d2dc79a7f07e'} Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:03:28.548456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:28.549595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:28.550769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Returning 404 to user: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:03:28.550769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-886c22d8-b75a-41f2-9c09-593e7f7b649c tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.052583 Oct 28 11:03:28.550769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 127/268] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:03:28 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e/os-volume_attachments => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:03:28.568462 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lazy-loading 'flavor' on Instance uuid 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:03:28.780298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-398ca0b9-4efc-4a42-a58e-5e087db4faca tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5/action" status: 200 len: 32 microversion: 2.1 time: 0.276904 Oct 28 11:03:28.785900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 142/269] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:03:28 2022] POST /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5/action => generated 32 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:03:28.792848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:28.793529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:28.805004 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.805535 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.806008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:28.807282 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:28.808123 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:28.810075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:29.109481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:29.123802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1e744b9e-8379-4dcc-aa13-4ac3132bd8eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.335954 Oct 28 11:03:29.124592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 128/270] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:28 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:29.932100 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:29.936006 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:29.942671 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:29.943158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:29.944349 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:29.945173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:29.945786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:29.946401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:29.960922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] HTTP exception thrown: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Request method failure captured: Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e HTTP/1.1 Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7aCeU30XGDIQxzF9h2t3eGZjB9CwS73-CeRmCBaa9AIZCfG4Gqlgx7HCtGRaxbTlBIdvJLIbbv_Y58TEWlh21c7HcQHmRfOXEiqi54OX2ueuEFc1mmWJb-rRfl65htdwFRXohPXbxq_fVL3_t4vMmZ4fkZdj1Kbc50PDnN66Xkr-DI Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:03:29.962737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 1903d1aaa3444181ac7b63234f4e8970 Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:29.964185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'bd2715aa-33ac-4197-90f4-d2dc79a7f07e'} Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:03:29.965154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:29.966393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:03:29.967561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Returning 404 to user: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:03:29.967561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-824cfc70-6548-470a-a415-a35da075ef9f tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "DELETE /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 404 len: 111 microversion: 2.1 time: 0.040406 Oct 28 11:03:29.967561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 143/271] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:03:29 2022] DELETE /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:03:29.972857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:29.973488 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] [instance: bd2715aa-33ac-4197-90f4-d2dc79a7f07e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:29.981460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:29.981946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:29.982762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:29.983436 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:29.984185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:29.984770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:30.000880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] HTTP exception thrown: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Request method failure captured: Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e HTTP/1.1 Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aCeU30XGDIQxzF9h2t3eGZjB9CwS73-CeRmCBaa9AIZCfG4Gqlgx7HCtGRaxbTlBIdvJLIbbv_Y58TEWlh21c7HcQHmRfOXEiqi54OX2ueuEFc1mmWJb-rRfl65htdwFRXohPXbxq_fVL3_t4vMmZ4fkZdj1Kbc50PDnN66Xkr-DI Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:03:30.003258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/115fe2408cba49bb8b84af24fc8ce604"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 115fe2408cba49bb8b84af24fc8ce604 Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-DeleteServersTestJSON-126995685 Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 1903d1aaa3444181ac7b63234f4e8970 Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-DeleteServersTestJSON-126995685-project Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:30.004853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'bd2715aa-33ac-4197-90f4-d2dc79a7f07e'} Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:03:30.005889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. Oct 28 11:03:30.007635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:30.008885 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] Returning 404 to user: Instance bd2715aa-33ac-4197-90f4-d2dc79a7f07e could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:03:30.008885 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7a758d2e-0fd8-4cbd-896a-04c1aa2d3843 tempest-DeleteServersTestJSON-126995685 tempest-DeleteServersTestJSON-126995685-project] 10.222.0.46 "GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e" status: 404 len: 111 microversion: 2.1 time: 0.035879 Oct 28 11:03:30.008885 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 129/272] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:29 2022] GET /compute/v2.1/servers/bd2715aa-33ac-4197-90f4-d2dc79a7f07e => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:03:30.153257 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:30.154392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:30.164936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:30.165836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:30.166508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:30.167436 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:30.168100 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:30.168747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:30.523061 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:30.535995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7d873840-cd7b-4755-8c8e-d27a6c168dda tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.387416 Oct 28 11:03:30.538024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 144/273] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:30 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:31.561162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:31.562853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:31.570673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:31.573158 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:31.577786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:31.580909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:31.581617 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:31.582695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:31.861434 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:31.870485 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9ed17ce3-bc6e-4e1d-9fe9-18e9bdb89536 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.313162 Oct 28 11:03:31.871332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 130/274] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:31 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:32.896511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:32.897319 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:32.905738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:32.906808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:32.907316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:32.908111 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:32.908663 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:32.909181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:33.210969 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:33.222688 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a9bbfcf5-14c8-4fa4-9ffd-960783c4c9d4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.330004 Oct 28 11:03:33.223341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 145/275] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:32 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:34.253529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:34.254171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:34.261861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:34.262385 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:34.262816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:34.278020 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:34.278020 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:34.278020 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:34.435832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:34.452268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-960185e9-7735-43ca-b183-d344ad2d37a7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.207660 Oct 28 11:03:34.453531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 131/276] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:34 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:35.477900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:35.478546 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:35.484741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:35.485216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:35.485645 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:35.486372 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:35.486818 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:35.487263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:35.655463 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:35.781923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:35.790722 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cf8b1446-d284-4107-86c7-92f86b7845cf tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.318195 Oct 28 11:03:35.791298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 146/277] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:35 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:36.812481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:36.813095 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:36.820039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:36.820508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:36.820945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:36.821635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:36.822121 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:36.822657 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:36.993126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:37.001241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a9882d91-0266-4cf5-abfb-08a89fd6e342 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.192566 Oct 28 11:03:37.002131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 132/278] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:36 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:38.025514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:38.026204 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:38.032425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:38.032874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:38.033309 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:38.034010 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:38.034468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:38.034890 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:38.176130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:38.186900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8d637294-9006-48f1-b7da-df0a11adc9e7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.162673 Oct 28 11:03:38.186900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 147/279] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:38 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:39.206309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:39.207561 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:39.214236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:39.214887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:39.215320 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:39.216017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:39.216465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:39.216884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:39.372033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:39.382069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fbaf3412-ee65-4d19-b545-d9bc09fe8c11 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.179298 Oct 28 11:03:39.386172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 133/280] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:39 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:40.405951 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:40.406889 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:40.414310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:40.414939 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:40.415548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:40.416395 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:40.417219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:40.417784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:40.577054 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:40.589212 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f6801ac9-212d-47d2-ba5f-d2a9bbb07aa7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.187310 Oct 28 11:03:40.590275 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 148/281] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:40 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:41.614569 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:41.615196 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:41.641633 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:41.642432 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:41.643102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:41.644033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:41.644812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:41.645449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:41.809478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:41.820422 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-03b219c1-c498-499a-a9f3-da4a496260f4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.211307 Oct 28 11:03:41.821482 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 134/282] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:41 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:42.874124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:42.874735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:42.889393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:42.890123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:42.890712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:42.891701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:42.892359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:42.893966 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:43.134257 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:43.142343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-39eb7a5b-1490-4ecb-b996-993f50be8d83 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.294029 Oct 28 11:03:43.144118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 149/283] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:42 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:44.168496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:44.169238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:44.175491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:44.176023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:44.176452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:44.177133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:44.177579 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:44.178052 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:44.317539 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:44.335026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2ffa085b-8208-4724-8073-f42bd78118a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.170930 Oct 28 11:03:44.335598 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 135/284] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:44 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:45.357996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:45.358628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:45.364721 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.365235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.365672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.366378 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.366833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.367256 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.489046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:45.497143 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5cca1c75-f7cb-4f4f-bc8c-cffb5c2ec2f1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1502 microversion: 2.1 time: 0.143054 Oct 28 11:03:45.497702 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 150/285] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:45 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1502 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:45.519695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0da314a1-d18d-44f8-81e5-452b4fcdf4c5"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:45.522105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:45.528302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.528781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.529211 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.529889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.530402 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.530822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.553262 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] HTTP exception thrown: Cannot 'attach_volume' instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 while it is in vm_state rescued Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Request method failure captured: Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: POST /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5/os-volume_attachments HTTP/1.1 Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 74 Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aIhPtnefCCqNmAPadWpk5584o2_2r0isTmVkRGKzWI-I7c1QbiFIdoruXOy5h0VZogX9L4sPi-DVMWkisxvUM6avn2ESWp6fMy2fmzi6DazAAqVi14U8Y2drfllWP-AiHWNP38WDQIVdY8aOS1D9QC35crpyrYbxvM8s51pB0LFgc Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:03:45.554851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 347e262907da4a83a197aaf42cecf51d Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {"volumeAttachment": {"volumeId": "0da314a1-d18d-44f8-81e5-452b4fcdf4c5"}} Oct 28 11:03:45.556415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Cannot 'attach_volume' instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 while it is in vm_state rescued Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'server_id': '54cf39f2-cfd4-4f12-bddb-36666db9e1d5', 'body': {'volumeAttachment': {'volumeId': '0da314a1-d18d-44f8-81e5-452b4fcdf4c5'}}} Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:03:45.557223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 while it is in vm_state rescued Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:03:45.558723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:03:45.559975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:03:45.559975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:03:45.559975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 while it is in vm_state rescued Oct 28 11:03:45.559975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:03:45.559975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Returning 409 to user: Cannot 'attach_volume' instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 while it is in vm_state rescued {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:03:45.559975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6b8297e2-2ddc-4497-b95f-3bb1557da905 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.040515 Oct 28 11:03:45.559975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 136/286] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:03:45 2022] POST /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5/os-volume_attachments => generated 152 bytes in 42 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Oct 28 11:03:45.565507 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:45.566067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:45.572589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.573070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.573509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.574225 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.574689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.575113 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.638383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1ee827b8-7906-42ac-8f1a-e8b6858f2cd4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5/action" status: 202 len: 0 microversion: 2.1 time: 0.076659 Oct 28 11:03:45.641738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 151/287] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:03:45 2022] POST /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5/action => generated 0 bytes in 78 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:03:45.647852 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:45.648469 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:45.654403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.654894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.655330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.656329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:45.656874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:45.657485 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:45.787023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:45.796074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0342d717-aa33-4a81-825b-e80f4c5e88ba tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1510 microversion: 2.1 time: 0.152725 Oct 28 11:03:45.796666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 137/288] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:45 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1510 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:46.820312 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:46.821097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:46.827230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:46.827824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:46.828330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:46.829090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:46.829544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:46.829966 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:46.985372 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:46.992913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-192a391b-c922-424c-948c-3687c3e84742 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1510 microversion: 2.1 time: 0.177426 Oct 28 11:03:46.993493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 152/289] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:46 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1510 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:47.220817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1211942316", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:47.288389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] validate_networks() for None {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:03:47.297858 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1810006610", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1118103513", "security_groups": [{"name": "tempest-secgroup-smoke-1525806302"}], "block_device_mapping_v2": [{"uuid": "0fadb5f1-7cd6-43db-a7e6-19f02c8df548", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "447ffcd3-d772-4c15-92cb-558dd17897d4"}]}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:47.447402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] validate_networks() for [('447ffcd3-d772-4c15-92cb-558dd17897d4', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:03:47.602449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:03:47.603102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:03:47.603767 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:03:47.604146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:03:47.604576 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:03:47.616538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:47.617320 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:47.618077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:47.619006 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:47.619786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:47.620325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:47.642563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Getting quotas for project cf9538379ffd44ee9a3765f8dea19810. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:03:47.648248 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Getting quotas for user e5a66fd6314e43a28696e7faec4eb1b6 and project cf9538379ffd44ee9a3765f8dea19810. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:03:47.661647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:03:47.664254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:03:47.783152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ed66fa08-f3ba-4272-9fb0-57b05d0db3ac tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.566122 Oct 28 11:03:47.784377 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 138/290] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:03:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 568 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:03:47.796216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-33958490-3c1d-4176-a877-952f5d01adf8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:47.797345 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-33958490-3c1d-4176-a877-952f5d01adf8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:47.804880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:47.911204 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-33958490-3c1d-4176-a877-952f5d01adf8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:47.920334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-33958490-3c1d-4176-a877-952f5d01adf8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1215 microversion: 2.1 time: 0.128260 Oct 28 11:03:47.921467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 139/291] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:47 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1215 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:48.018760 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:48.019625 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:48.027703 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:48.028199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:48.028635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:48.031844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:48.032332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:48.032772 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:48.256954 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:03:48.259527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:03:48.260259 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:03:48.260987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:03:48.261610 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:03:48.275477 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:48.280288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:48.281293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:48.281825 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:48.282763 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:48.283255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:48.283706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:48.285710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a5661a64-1ee4-4fd3-b593-f3b964e7d670 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1510 microversion: 2.1 time: 0.272213 Oct 28 11:03:48.302244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 140/292] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:48 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1510 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:48.304421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:03:48.308869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for user 6b926f7161c0487a8c2b04ada3b374e3 and project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:03:48.377639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:03:48.381393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='0fadb5f1-7cd6-43db-a7e6-19f02c8df548',volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:03:48.551076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5f54df93-105a-4f59-8058-37132dbb7e3c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.256916 Oct 28 11:03:48.551995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 153/293] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:03:47 2022] POST /compute/v2.1/servers => generated 406 bytes in 1258 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 11:03:48.563279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1cd2bd8c-ec24-4a48-83c3-96d9bcefb288 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:48.563983 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1cd2bd8c-ec24-4a48-83c3-96d9bcefb288 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:48.621502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1cd2bd8c-ec24-4a48-83c3-96d9bcefb288 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:48.629374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1cd2bd8c-ec24-4a48-83c3-96d9bcefb288 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1095 microversion: 2.1 time: 0.069875 Oct 28 11:03:48.630518 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 141/294] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:48 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1095 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:48.948182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:48.949220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:48.962198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:48.962709 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:48.963152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:48.963869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:48.968805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:48.970813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:49.065295 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:49.074245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-13ff8069-9dc0-40d8-b2ff-47e376191d3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1211 microversion: 2.1 time: 0.132412 Oct 28 11:03:49.075037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 154/295] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:48 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1211 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:49.310102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:49.310744 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:49.321329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:49.322193 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:49.324928 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:49.326964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:49.329877 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:49.337674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:49.548943 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:49.560675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-42c6c8d5-c610-4082-9e1d-c3e4bad31822 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1516 microversion: 2.1 time: 0.255147 Oct 28 11:03:49.563160 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 142/296] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:49 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1516 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:49.652205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:49.652949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:49.660598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:49.661135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:49.661751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:49.663284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:49.663879 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:49.664446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:49.773395 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:49.784990 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4e3c06a2-b3fd-4c91-bdb1-f0d4dc21befc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1137 microversion: 2.1 time: 0.135589 Oct 28 11:03:49.797161 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 155/297] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:49 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1137 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:50.100959 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:50.102816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:50.126731 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:50.132030 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:50.133255 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:50.136136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:50.136925 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:50.139815 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:50.158325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:50.567061 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:50.595236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c5a72e78-a415-491a-82f1-a99b4a2783fd tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1285 microversion: 2.1 time: 0.500963 Oct 28 11:03:50.595887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 143/298] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:50 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1285 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:50.812263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:50.812847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:50.827880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:50.828365 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:50.828815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:50.832908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:50.836049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:50.838776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:51.214323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-469a170b-78e2-43bd-9545-ce4bfe1daae7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'create', calling method: .version_select at 0x7f42fa788e50>, body: {"keypair": {"name": "tempest-keypair-189909379", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNWvQMjsW1kwJUS8dGpwmHQZDbDTP6tH0U+xrrVNZMNEyauC6uSuqiqUottBthNunB5LR80ypF7+Ide368q6D+wwRAytQhvL3F0puS9qeltdFByfbPqEPz1IsscunQ4vgQ=="}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:51.264273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-469a170b-78e2-43bd-9545-ce4bfe1daae7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for project f1f49235fbe64b79bfef2cf828993c56. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:03:51.280936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:51.283904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-469a170b-78e2-43bd-9545-ce4bfe1daae7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for user 347e262907da4a83a197aaf42cecf51d and project f1f49235fbe64b79bfef2cf828993c56. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:03:51.291922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5cc9bfae-de81-4d29-8e06-a0335eee457a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1201 microversion: 2.1 time: 0.483410 Oct 28 11:03:51.292663 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 156/299] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:50 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1201 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:51.348253 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-469a170b-78e2-43bd-9545-ce4bfe1daae7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.138309 Oct 28 11:03:51.355707 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 144/300] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:03:51 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:03:51.625315 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:51.626945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:51.629852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:51.642227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:51.643767 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:51.644815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:51.646677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:51.648325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:51.650221 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:52.083236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:52.097654 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c66f9033-a1f2-4a08-a4b5-b07fdfe1b973 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.482384 Oct 28 11:03:52.102581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 157/301] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:51 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:52.321283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:52.322454 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:52.332483 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:52.333225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:52.334389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:52.336522 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:52.337862 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:52.338316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:52.902637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:52.912232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-947ccc1c-fbc6-4449-aabe-1fed00a9ff07 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1279 microversion: 2.1 time: 0.595091 Oct 28 11:03:52.913068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 145/302] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:52 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1279 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:53.015227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-945a8b29-5c14-49cf-9f64-bd5696cfd7ae req-d585a263-e9ea-4c8a-98f4-7cdb08efd532 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "892ed06b-4556-4ce0-843f-1a326bfe31d0", "tag": "491f350d-a116-4d79-b520-ef9f1b301107"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:53.056726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-945a8b29-5c14-49cf-9f64-bd5696cfd7ae req-d585a263-e9ea-4c8a-98f4-7cdb08efd532 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:53.058396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-945a8b29-5c14-49cf-9f64-bd5696cfd7ae req-d585a263-e9ea-4c8a-98f4-7cdb08efd532 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:53.059977 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-945a8b29-5c14-49cf-9f64-bd5696cfd7ae req-d585a263-e9ea-4c8a-98f4-7cdb08efd532 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:53.076078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-945a8b29-5c14-49cf-9f64-bd5696cfd7ae req-d585a263-e9ea-4c8a-98f4-7cdb08efd532 service nova] Creating event network-changed:491f350d-a116-4d79-b520-ef9f1b301107 for instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 on hv-cinder-zed-3 Oct 28 11:03:53.083129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-945a8b29-5c14-49cf-9f64-bd5696cfd7ae req-d585a263-e9ea-4c8a-98f4-7cdb08efd532 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090527 Oct 28 11:03:53.084274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 158/303] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:03:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:03:53.127213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:53.129259 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:53.141872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:53.142483 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:53.142915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:53.143745 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:53.144365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:53.144794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:53.489033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:53.507223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dc607966-12df-4a00-b964-9e4e797ae00f tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.384926 Oct 28 11:03:53.513103 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 146/304] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:53 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:53.939770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:53.940486 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:53.946667 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:53.947366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:53.947801 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:53.948491 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:53.949135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:53.959875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:54.118683 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:54.129228 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8c177895-6fd1-4e72-9fda-7001b5d2aaf6 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.193774 Oct 28 11:03:54.133508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 159/305] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:53 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:54.544284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-b12c88d6-4469-4264-9108-aecfba04f1a4 req-2ff46a2c-e003-408c-877f-f891b13d124b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "917e11e6-71de-4156-bffc-7a5a81135795", "tag": "4fc47c92-3eb6-4a2b-85be-a8582c825e05"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:54.544924 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:54.547525 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:54.563524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:54.566148 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:54.566666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:54.570329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-b12c88d6-4469-4264-9108-aecfba04f1a4 req-2ff46a2c-e003-408c-877f-f891b13d124b service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:54.570862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:54.571576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:54.572245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:54.577626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-b12c88d6-4469-4264-9108-aecfba04f1a4 req-2ff46a2c-e003-408c-877f-f891b13d124b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:54.578386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-b12c88d6-4469-4264-9108-aecfba04f1a4 req-2ff46a2c-e003-408c-877f-f891b13d124b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:54.605123 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-b12c88d6-4469-4264-9108-aecfba04f1a4 req-2ff46a2c-e003-408c-877f-f891b13d124b service nova] Creating event network-changed:4fc47c92-3eb6-4a2b-85be-a8582c825e05 for instance 917e11e6-71de-4156-bffc-7a5a81135795 on hv-cinder-zed-3 Oct 28 11:03:54.616555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-b12c88d6-4469-4264-9108-aecfba04f1a4 req-2ff46a2c-e003-408c-877f-f891b13d124b service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085210 Oct 28 11:03:54.618798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 147/306] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:03:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:03:54.754822 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:54.766611 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-425fb262-705c-469d-b2dc-74a84d0887c9 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.233697 Oct 28 11:03:54.768513 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 160/307] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:54 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:55.154978 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:55.155672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:55.162333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:55.163102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:55.163711 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:55.164578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:55.165172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:55.165758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:55.305841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1339118967", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "57d99b3c-83bf-49e2-81d2-c12ed26cdb71"}], "security_groups": [{"name": "tempest-securitygroup--1576328441"}], "key_name": "tempest-keypair-189909379", "user_data": "}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:03:55.325392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:55.336751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e3ac1649-5def-47e6-9e00-4813142317f2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.185270 Oct 28 11:03:55.337472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 148/308] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:55 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:55.422194 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] validate_networks() for [('57d99b3c-83bf-49e2-81d2-c12ed26cdb71', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:03:55.799330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:55.800724 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:55.807873 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:55.808732 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:55.809344 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:55.810307 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:55.811149 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:55.811519 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:55.995916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:56.007009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0a25674d-3c12-4891-89d2-bcef65f5a155 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.212031 Oct 28 11:03:56.008318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 149/309] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:55 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:03:56.174539 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:03:56.174936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:03:56.175411 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:03:56.175819 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:03:56.176230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:03:56.194683 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:56.195510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:56.195972 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:56.196698 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:56.197153 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:56.197576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:56.217642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for project f1f49235fbe64b79bfef2cf828993c56. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:03:56.223416 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Getting quotas for user 347e262907da4a83a197aaf42cecf51d and project f1f49235fbe64b79bfef2cf828993c56. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:03:56.236809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:03:56.242892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:03:56.313066 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f96b2551-62d3-4e0b-8a10-2b14db159ff1 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.014047 Oct 28 11:03:56.315748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 161/310] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:03:55 2022] POST /compute/v2.1/servers => generated 409 bytes in 1017 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 11:03:56.328474 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-86ad5ae9-0755-47c2-9003-3fd4812e5155 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:56.330730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-86ad5ae9-0755-47c2-9003-3fd4812e5155 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:56.360639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:56.361510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:56.371982 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:56.378464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:56.379242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:56.380128 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:56.380649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:56.381079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:56.458368 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-86ad5ae9-0755-47c2-9003-3fd4812e5155 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:56.467947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-86ad5ae9-0755-47c2-9003-3fd4812e5155 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1245 microversion: 2.1 time: 0.144214 Oct 28 11:03:56.468532 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 150/311] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:56 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1245 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:56.585628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:56.598685 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a6d280cb-65c2-46f3-a0fd-897a746dd012 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.242069 Oct 28 11:03:56.599820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 162/312] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:56 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:03:57.036097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:57.038451 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:57.049621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:57.050759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:57.053404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:57.055363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:57.056173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:57.057593 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:57.257531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:57.272909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d85e92d1-f416-4805-bbf2-2699c0d44bd2 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.239088 Oct 28 11:03:57.273838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 151/313] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:57 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:57.494908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:57.495517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:57.501303 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:57.502105 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:57.502612 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:57.503574 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:57.504266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:57.504723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:57.580488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:57.589007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c8c73861-f2dd-45f3-8533-f8d18ca1185c tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1249 microversion: 2.1 time: 0.097958 Oct 28 11:03:57.589822 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 163/314] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:57 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1249 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:57.622316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:57.622923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:57.635308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:57.637280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:57.637897 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:57.638409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:57.640304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:57.640790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:57.641221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:57.784969 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:57.794016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-30a307c0-c4db-49ba-9083-cf247714b6b9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.175981 Oct 28 11:03:57.816774 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 152/315] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:57 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:58.295755 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:58.296607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:58.303611 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:58.304311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:58.304782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:58.305583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:58.307035 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:58.307564 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:58.312230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:03:58.445800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:58.453511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-89d5b296-36d5-48b5-bbc8-0761d554d011 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.161663 Oct 28 11:03:58.454093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 164/316] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:58 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:58.612097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:58.612819 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:58.620064 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:58.621157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:58.621855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:58.622582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:58.623040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:58.623583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:58.687337 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:58.694981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4d37e6a2-3988-4aca-9aca-7d410948838f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1241 microversion: 2.1 time: 0.087003 Oct 28 11:03:58.695534 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 153/317] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:58 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1241 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:58.820469 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:58.821161 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:58.827442 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:58.828015 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:58.828492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:58.829169 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:58.829716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:58.830157 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:58.962500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:58.970566 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-aebc4021-e673-48e6-8385-c7a8fbb6c271 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.153978 Oct 28 11:03:58.971119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 165/318] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:58 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:59.476270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:59.476887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:59.482769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:59.483251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:59.483749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:59.484701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:59.485155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:59.485578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:59.613628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:59.621462 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0e364d2e-bb5e-462c-a641-0d820e9cbb90 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.149156 Oct 28 11:03:59.622595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 154/319] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:59 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:59.717685 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:59.718263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:03:59.724761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:59.725281 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:59.725656 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:59.726360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:03:59.726810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:03:59.727226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:03:59.788847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:03:59.796809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-70743e8d-3163-40e7-85fd-43d4c3c5fd6d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1241 microversion: 2.1 time: 0.082983 Oct 28 11:03:59.797369 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 166/320] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:59 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1241 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:03:59.993939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:03:59.994582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:00.000368 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:00.000830 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:00.001266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:00.001948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:00.002455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:00.002906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:00.137505 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:00.150497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7a6394fe-bf00-4b9c-aef7-dd2046e82233 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.160198 Oct 28 11:04:00.151109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 155/321] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:03:59 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:00.644770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:00.645587 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:00.652216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:00.652753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:00.653313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:00.654370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:00.654865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:00.655292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:00.782777 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:00.790488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d503abf9-a7ea-4f60-b7fe-cecc1354c296 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.149768 Oct 28 11:04:00.791048 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 167/322] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:00 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:00.820342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:00.820974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:00.832357 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:00.832885 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:00.833501 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:00.834060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:00.834511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:00.834954 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:00.899056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:00.906599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9a81a49a-261f-4d2b-a7ae-edf5f17e3185 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1297 microversion: 2.1 time: 0.091339 Oct 28 11:04:00.907158 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 156/323] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:00 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1297 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:01.173583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:01.174287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:01.181323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:01.181807 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:01.182268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:01.182965 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:01.183422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:01.183838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:01.323230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:01.334350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fdb4c08b-5851-4ea4-b18d-7197829fe138 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.163608 Oct 28 11:04:01.334931 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 168/324] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:01 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:01.816458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:01.817477 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:01.824914 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:01.825478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:01.825787 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:01.826578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:01.827106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:01.827710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:01.929353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:01.929963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:01.936238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:01.936643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:01.937070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:01.937757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:01.938230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:01.938648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:01.970597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:01.982351 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c6668f15-e151-4c6c-ac9a-263fb2d3b858 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.170160 Oct 28 11:04:01.983183 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 157/325] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:01 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:02.004185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:02.012945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4a7ffe8f-0e81-44f7-ae2c-2fd3afa0cbc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1297 microversion: 2.1 time: 0.087449 Oct 28 11:04:02.013520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 169/326] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:01 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1297 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:02.358221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:02.358976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:02.365999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:02.366489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:02.366922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:02.367617 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:02.368067 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:02.368482 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:02.519233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:02.526958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-60c243ab-2ad3-4720-ad87-04b3a511c218 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.172779 Oct 28 11:04:02.527516 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 158/327] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:02 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:02.937181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-aee7144d-1294-4f6f-b8a2-e34d1cee0c6a req-4540e31b-0d6d-412b-8aab-6af9193b87a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "892ed06b-4556-4ce0-843f-1a326bfe31d0", "name": "network-vif-plugged", "status": "completed", "tag": "491f350d-a116-4d79-b520-ef9f1b301107"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:02.948393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-aee7144d-1294-4f6f-b8a2-e34d1cee0c6a req-4540e31b-0d6d-412b-8aab-6af9193b87a6 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:02.948869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-aee7144d-1294-4f6f-b8a2-e34d1cee0c6a req-4540e31b-0d6d-412b-8aab-6af9193b87a6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:02.949302 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-aee7144d-1294-4f6f-b8a2-e34d1cee0c6a req-4540e31b-0d6d-412b-8aab-6af9193b87a6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:02.964320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-aee7144d-1294-4f6f-b8a2-e34d1cee0c6a req-4540e31b-0d6d-412b-8aab-6af9193b87a6 service nova] Creating event network-vif-plugged:491f350d-a116-4d79-b520-ef9f1b301107 for instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 on hv-cinder-zed-3 Oct 28 11:04:02.971696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-aee7144d-1294-4f6f-b8a2-e34d1cee0c6a req-4540e31b-0d6d-412b-8aab-6af9193b87a6 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038261 Oct 28 11:04:02.972284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 170/328] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:03.012349 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:03.013246 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:03.024415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:03.025050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:03.026325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:03.027748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:03.028373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:03.029707 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:03.037910 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:03.038950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:03.054481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:03.055297 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:03.056765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:03.059361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:03.061499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:03.062311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:03.271559 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:03.283115 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-21f77c7c-b2e8-4310-b16d-e6a91bdf665a tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.279826 Oct 28 11:04:03.287277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 159/329] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:03 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:03.380050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:03.387728 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-eeceb8c5-7bd3-4f0f-90a6-2ac2e0422ac8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.355322 Oct 28 11:04:03.388293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 171/330] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:03 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:03.551607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:03.552259 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:03.558358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:03.559072 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:03.559668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:03.560554 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:03.561167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:03.561591 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:03.726491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:03.746229 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ffea4990-2d37-436d-b6fe-83bd2305e3c9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.198478 Oct 28 11:04:03.747192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 160/331] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:03 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:03.841613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-c5163534-d4fe-4d76-952f-9a213a1a6ce2 req-ff2268f8-4e55-4eb3-b05b-aefa26dc17cd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01a19441-142f-41aa-9253-c5b50c929e93", "tag": "eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:03.851920 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c5163534-d4fe-4d76-952f-9a213a1a6ce2 req-ff2268f8-4e55-4eb3-b05b-aefa26dc17cd service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:03.852618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c5163534-d4fe-4d76-952f-9a213a1a6ce2 req-ff2268f8-4e55-4eb3-b05b-aefa26dc17cd service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:03.853227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c5163534-d4fe-4d76-952f-9a213a1a6ce2 req-ff2268f8-4e55-4eb3-b05b-aefa26dc17cd service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:03.866586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-c5163534-d4fe-4d76-952f-9a213a1a6ce2 req-ff2268f8-4e55-4eb3-b05b-aefa26dc17cd service nova] Creating event network-changed:eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f for instance 01a19441-142f-41aa-9253-c5b50c929e93 on hv-cinder-zed-3 Oct 28 11:04:03.871299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-c5163534-d4fe-4d76-952f-9a213a1a6ce2 req-ff2268f8-4e55-4eb3-b05b-aefa26dc17cd service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034138 Oct 28 11:04:03.871893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 172/332] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:04.311113 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:04.312360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:04.318521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:04.318996 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:04.319810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:04.320801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:04.321256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:04.321680 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:04.417523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:04.417523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:04.433499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:04.434144 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:04.434631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:04.435339 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:04.435799 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:04.436369 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:04.497244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:04.512961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-13d4928c-cd60-41c8-aca7-74d3402d211e tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1315 microversion: 2.1 time: 0.206402 Oct 28 11:04:04.513949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 161/333] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:04 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1315 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:04.721372 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:04.730713 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1653a552-7168-43d4-85ab-f132489705f0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.322341 Oct 28 11:04:04.734013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 173/334] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:04 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:04.768758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:04.769373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:04.775722 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:04.776209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:04.776637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:04.777488 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:04.777949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:04.778398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:04.794790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:04.957146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:04.966606 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-82ff10e8-0894-4212-b228-c29664fbb686 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.201625 Oct 28 11:04:04.967270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 162/335] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:04 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:05.538720 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:05.541648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:05.556373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:05.557620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:05.558466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:05.559374 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:05.560574 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:05.561190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:05.756461 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:05.758283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:05.766238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:05.767453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:05.767998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:05.768796 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:05.769448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:05.769637 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:05.770343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:05.784066 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6da2874d-a2d3-4786-8864-6c85a4bfd0f8 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1464 microversion: 2.1 time: 0.250578 Oct 28 11:04:05.784650 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 174/336] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:05 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1464 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:05.834521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:05.835097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:05.841804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:05.842265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:05.842699 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:05.843395 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:05.843849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:05.844622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:05.891783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:04:05.955784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:05.967774 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9e4ed180-a764-48ba-921f-9b66a918f2e4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.215780 Oct 28 11:04:05.968429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 163/337] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:05 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:05.991377 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:05.992417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:05.994410 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5432184d-df29-4558-83fe-61b5cc905a3c tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "DELETE /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 204 len: 0 microversion: 2.1 time: 0.163562 Oct 28 11:04:05.995008 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 175/338] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:04:05 2022] DELETE /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 11:04:06.000163 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:06.000903 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:06.001517 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:06.002859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:06.003781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:06.004784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:06.007580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:06.008552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:06.019068 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:06.019747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:06.020386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:06.021791 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:06.022278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:06.022700 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:06.191811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:06.202824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1b8c40db-234e-443f-8fd1-b1cff0229201 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.216454 Oct 28 11:04:06.203399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 164/339] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:05 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:06.206124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:06.213048 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:06.216954 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ddb4bbc5-fd8a-4ff3-9f49-9b4a2d12c8ca tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1470 microversion: 2.1 time: 0.216778 Oct 28 11:04:06.224653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 176/340] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:05 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1470 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:06.999272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:06.999891 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:07.003573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=16 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:07.010866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:07.011565 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:07.012219 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:07.013138 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:07.013916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:07.014558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:07.186531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:07.194545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0b0e2d1d-f8a7-4f93-8bc6-874b78229594 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.207256 Oct 28 11:04:07.195426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 165/341] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:06 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:07.227465 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:07.228674 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:07.230645 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=12 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:07.238385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:07.239422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:07.240373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:07.241390 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:07.242158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:07.242732 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:07.245574 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:07.255270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:07.262544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:07.263031 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:07.263463 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:07.264175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:07.264625 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:07.265048 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:07.428889 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:07.437697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d26d27a9-a27b-4346-b113-976600003168 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.214757 Oct 28 11:04:07.439031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 177/342] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:07 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:07.453317 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:07.464748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d9aa8587-5e78-48a3-975e-6bc7c5f2be29 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1470 microversion: 2.1 time: 0.227395 Oct 28 11:04:07.465335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 166/343] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:07 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1470 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:08.219330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:08.219921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:08.226770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:08.227311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:08.227750 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:08.228469 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:08.228927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:08.229355 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:08.360576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:08.369057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2bbf5cb5-70b3-4827-b2b1-f3872fc4503e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.153615 Oct 28 11:04:08.369626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 178/344] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:08 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:08.464534 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:08.465195 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:08.471919 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:08.472325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:08.472753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:08.473439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:08.474441 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:08.474930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:08.490985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:08.491592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:08.497359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:08.497839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:08.498323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:08.499017 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:08.499467 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:08.499887 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:08.612263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:08.620852 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6eae592f-df81-4c9a-a632-968e170e6ac5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.160115 Oct 28 11:04:08.621436 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 167/345] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:08 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:08.636396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:08.644210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-28a1d770-864f-487c-a8a1-f471de88a749 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1470 microversion: 2.1 time: 0.157647 Oct 28 11:04:08.644766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 179/346] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:08 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1470 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:09.392078 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:09.392611 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:09.399358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:09.399833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:09.400260 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:09.400941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:09.401385 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:09.401802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:09.530468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:09.538762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-46e29baa-611c-4269-865c-a63c0ae0cfb8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.150602 Oct 28 11:04:09.539354 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 168/347] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:09 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:09.644622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:09.645236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:09.651755 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:09.652228 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:09.652653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:09.653347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:09.653796 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:09.654235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:09.667331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:09.668085 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:09.674153 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:09.677096 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:09.680397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:09.680875 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:09.681605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:09.682101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:09.682596 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:09.797293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:09.804995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-afdda7f2-bf63-497c-9ade-644084d8fd6e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.164314 Oct 28 11:04:09.805572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 180/348] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:09 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:09.815648 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:09.823933 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4943fa09-ef2a-481e-bdc8-581b98dd6545 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1470 microversion: 2.1 time: 0.160427 Oct 28 11:04:09.824491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 169/349] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:09 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1470 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:10.561708 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:10.562860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:10.571292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:10.572069 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:10.572709 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:10.573424 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:10.573886 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:10.574379 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:10.735160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:10.743823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7585d50c-9aab-4f24-8b0c-f5e7c400e0ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.186176 Oct 28 11:04:10.744675 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 181/350] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:10 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:10.828427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:10.829041 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:10.837752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:10.838264 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:10.838706 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:10.839412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:10.839930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:10.840331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:10.852464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:10.853601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:10.863577 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:10.865046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:10.866689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:10.867751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:10.868457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:10.869418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:11.080455 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:11.088656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:11.089268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-52df95b6-bc9f-4566-9564-b1f728c633c5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1470 microversion: 2.1 time: 0.244636 Oct 28 11:04:11.090059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 182/351] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:10 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1470 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:11.096749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-18916d20-3b1f-40ed-b578-2d937e0679b0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.272395 Oct 28 11:04:11.110767 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 170/352] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:10 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:11.768457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:11.768996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:11.775444 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:11.775910 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:11.776334 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:11.777002 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:11.777448 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:11.777862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:11.904790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:11.913013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ce9a2882-86e1-470c-8142-bb61d9dddb92 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.148499 Oct 28 11:04:11.913572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 183/353] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:11 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:12.113752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:12.115722 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:12.120803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:12.123261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:12.123786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:12.124568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:12.125152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:12.125645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:12.126351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:12.127199 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:12.135886 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:12.136471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:12.136974 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:12.137871 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:12.138368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:12.138787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:12.284163 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:12.295682 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4de53218-ad03-477c-a4cf-f316a0e39dd5 tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1470 microversion: 2.1 time: 0.186153 Oct 28 11:04:12.296640 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 171/354] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:12 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1470 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:12.604960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:12.615442 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7b17f596-f613-4638-9305-bdde721d1d3d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.493321 Oct 28 11:04:12.616031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 184/355] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:12 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:12.776751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-b93c6ab3-a239-4ebd-8429-580758774b8d req-3bfe3e3f-dfc9-42b6-a8f4-11b721fe64e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "892ed06b-4556-4ce0-843f-1a326bfe31d0", "name": "network-vif-deleted", "tag": "491f350d-a116-4d79-b520-ef9f1b301107"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:12.796218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-b93c6ab3-a239-4ebd-8429-580758774b8d req-3bfe3e3f-dfc9-42b6-a8f4-11b721fe64e3 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:12.797892 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-b93c6ab3-a239-4ebd-8429-580758774b8d req-3bfe3e3f-dfc9-42b6-a8f4-11b721fe64e3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:12.799604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-b93c6ab3-a239-4ebd-8429-580758774b8d req-3bfe3e3f-dfc9-42b6-a8f4-11b721fe64e3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:12.817546 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-b93c6ab3-a239-4ebd-8429-580758774b8d req-3bfe3e3f-dfc9-42b6-a8f4-11b721fe64e3 service nova] Creating event network-vif-deleted:491f350d-a116-4d79-b520-ef9f1b301107 for instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 on hv-cinder-zed-3 Oct 28 11:04:12.825247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-b93c6ab3-a239-4ebd-8429-580758774b8d req-3bfe3e3f-dfc9-42b6-a8f4-11b721fe64e3 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051875 Oct 28 11:04:12.826889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 172/356] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:12.935589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:12.936193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:12.942297 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:12.942811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:12.943244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:12.943934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:12.944384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:12.944801 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:13.090880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:13.101038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0f5360f5-15f5-4e59-a5fa-991880fd957d tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.169277 Oct 28 11:04:13.101961 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 185/357] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:12 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:13.104334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-c083a3d1-0e00-4a16-8b39-f4f7aa81a059 req-e7064c8a-fa3e-4e2c-91a1-644a23b047c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01a19441-142f-41aa-9253-c5b50c929e93", "name": "network-vif-plugged", "status": "completed", "tag": "eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:13.118685 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-c083a3d1-0e00-4a16-8b39-f4f7aa81a059 req-e7064c8a-fa3e-4e2c-91a1-644a23b047c3 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:13.119969 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-c083a3d1-0e00-4a16-8b39-f4f7aa81a059 req-e7064c8a-fa3e-4e2c-91a1-644a23b047c3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:13.120673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-c083a3d1-0e00-4a16-8b39-f4f7aa81a059 req-e7064c8a-fa3e-4e2c-91a1-644a23b047c3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:13.135523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-c083a3d1-0e00-4a16-8b39-f4f7aa81a059 req-e7064c8a-fa3e-4e2c-91a1-644a23b047c3 service nova] Creating event network-vif-plugged:eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f for instance 01a19441-142f-41aa-9253-c5b50c929e93 on hv-cinder-zed-3 Oct 28 11:04:13.140588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-c083a3d1-0e00-4a16-8b39-f4f7aa81a059 req-e7064c8a-fa3e-4e2c-91a1-644a23b047c3 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042892 Oct 28 11:04:13.141366 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 173/358] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:04:13.319639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:13.320184 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:13.326116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:13.326597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:13.327030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:13.327710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:13.328169 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:13.328592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:13.357656 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:13.358192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:13.358625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:13.359063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lazy-loading 'fault' on Instance uuid 892ed06b-4556-4ce0-843f-1a326bfe31d0 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:04:13.403733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5a66fd6314e43a28696e7faec4eb1b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf9538379ffd44ee9a3765f8dea19810', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:13.411979 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-210fd84d-5fcc-4548-96f2-4e2e791e2b3d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 200 len: 1305 microversion: 2.1 time: 0.096843 Oct 28 11:04:13.412921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 186/359] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:13 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 1305 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:13.640261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:13.641249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:13.647708 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:13.648194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:13.648625 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:13.649308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:13.649755 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:13.650200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:13.820217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:13.830750 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7422172d-ad70-4fce-8df5-db9a5a6e0f55 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.194603 Oct 28 11:04:13.831679 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 174/360] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:13 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:14.125392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:14.126364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:14.133090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:14.133608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:14.134108 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:14.134912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:14.135531 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:14.136100 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:14.269146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:14.277640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d45caf85-eeae-4727-a794-b50073051f34 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.156078 Oct 28 11:04:14.278247 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 187/361] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:14 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:14.435104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:14.435630 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] [instance: 892ed06b-4556-4ce0-843f-1a326bfe31d0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:14.441908 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:14.442910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:14.443336 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:14.444009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:14.444453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:14.444866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:14.457804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] HTTP exception thrown: Instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 could not be found. Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Request method failure captured: Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 HTTP/1.1 Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7cQqn9StajwqUC4mugrwN0zkgePZQP0JKKmzFNqk1Mzmrlhgz_Zm1o3a4mELbEk2QY6z_eVfdUD-aUWZZG7hyMu2F46Njl9ItcGl7X-Pxl-zErm01Q3CZ9OAoVj9afXaQ7OGJmr1LPkLfCVuhRkUl_cR-COvOqZmoUbAV3tsutny4o Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: cf9538379ffd44ee9a3765f8dea19810 Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-VolumesNegativeTest-1080108375 Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:04:14.460069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/cf9538379ffd44ee9a3765f8dea19810"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_cf9538379ffd44ee9a3765f8dea19810"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/cf9538379ffd44ee9a3765f8dea19810"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/cf9538379ffd44ee9a3765f8dea19810"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-VolumesNegativeTest-1080108375 Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: cf9538379ffd44ee9a3765f8dea19810 Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-VolumesNegativeTest-1080108375 Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-VolumesNegativeTest-1080108375-project Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: e5a66fd6314e43a28696e7faec4eb1b6 Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-VolumesNegativeTest-1080108375-project Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 could not be found. Oct 28 11:04:14.461638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '892ed06b-4556-4ce0-843f-1a326bfe31d0'} Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 could not be found. Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:04:14.462502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 could not be found. Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 could not be found. Oct 28 11:04:14.463946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:04:14.465323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] Returning 404 to user: Instance 892ed06b-4556-4ce0-843f-1a326bfe31d0 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:04:14.465323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-974b9f45-1b4a-46f4-af6f-083a792c887d tempest-VolumesNegativeTest-1080108375 tempest-VolumesNegativeTest-1080108375-project] 10.222.0.46 "GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0" status: 404 len: 111 microversion: 2.1 time: 0.030335 Oct 28 11:04:14.465323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 175/362] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:14 2022] GET /compute/v2.1/servers/892ed06b-4556-4ce0-843f-1a326bfe31d0 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:04:14.856013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:14.856626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:14.863609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:14.864097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:14.865058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:14.865774 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:14.866341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:14.866766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:15.018694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:15.028055 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9ba0ece5-ecd5-4a82-a91c-3faf7c7851cf tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.175814 Oct 28 11:04:15.041997 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 188/363] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:14 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:15.165464 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-acf2e380-c125-4b5e-a117-4211031f014c req-59e6f96e-ed35-41b9-8985-28ab0634a947 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "917e11e6-71de-4156-bffc-7a5a81135795", "name": "network-vif-plugged", "status": "completed", "tag": "4fc47c92-3eb6-4a2b-85be-a8582c825e05"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:15.178302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-acf2e380-c125-4b5e-a117-4211031f014c req-59e6f96e-ed35-41b9-8985-28ab0634a947 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:15.178784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-acf2e380-c125-4b5e-a117-4211031f014c req-59e6f96e-ed35-41b9-8985-28ab0634a947 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:15.179248 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-acf2e380-c125-4b5e-a117-4211031f014c req-59e6f96e-ed35-41b9-8985-28ab0634a947 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:15.196794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-acf2e380-c125-4b5e-a117-4211031f014c req-59e6f96e-ed35-41b9-8985-28ab0634a947 service nova] Creating event network-vif-plugged:4fc47c92-3eb6-4a2b-85be-a8582c825e05 for instance 917e11e6-71de-4156-bffc-7a5a81135795 on hv-cinder-zed-3 Oct 28 11:04:15.202153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-acf2e380-c125-4b5e-a117-4211031f014c req-59e6f96e-ed35-41b9-8985-28ab0634a947 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041811 Oct 28 11:04:15.203365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 176/364] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:15.304627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:15.305314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:15.311371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:15.312189 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:15.312616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:15.313622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:15.314390 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:15.314788 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:15.473082 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:15.481590 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3d67e42a-8cb4-4a54-8281-7cfa18049b19 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1371 microversion: 2.1 time: 0.181783 Oct 28 11:04:15.482198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 189/365] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:15 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1371 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:16.054417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:16.055130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:16.061523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:16.062309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:16.062941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:16.063842 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:16.064494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:16.065109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:16.239413 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:16.258347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6f26aade-6238-4e0b-b857-236b0ab934d1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.209321 Oct 28 11:04:16.259483 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 177/366] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:16 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:16.505481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:16.506105 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:16.513634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:16.514209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:16.515027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:16.519745 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:16.521261 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:16.521926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:16.729460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:16.739552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2d93841e-88d3-4799-a04a-cadfaf247036 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1564 microversion: 2.1 time: 0.238344 Oct 28 11:04:16.740783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 190/367] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:16 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1564 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:16.762998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:16.763493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:16.771641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:16.772190 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:16.772748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:16.773510 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:16.774092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:16.775189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:16.874404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a7bfcd4e-49b8-42f1-b63a-97a9c7305b77 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.115392 Oct 28 11:04:16.874984 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 178/368] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:04:16 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-interface => generated 284 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:17.282275 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:17.282968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:17.289218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:17.289707 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:17.290185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:17.290880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:17.291339 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:17.291765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:17.572505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:17.586649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cbb4d388-99f6-4b3f-8cc9-a098ede85ce1 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1267 microversion: 2.1 time: 0.308687 Oct 28 11:04:17.587494 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 191/369] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:17 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1267 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:18.415826 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-2a42e29e-3fbc-422a-9ad1-25f9578d6e9e req-31cb156e-e2bd-41ac-8766-e09eeeaacf82 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01a19441-142f-41aa-9253-c5b50c929e93", "tag": "eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:18.426244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:18.430561 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-2a42e29e-3fbc-422a-9ad1-25f9578d6e9e req-31cb156e-e2bd-41ac-8766-e09eeeaacf82 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:18.431033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-2a42e29e-3fbc-422a-9ad1-25f9578d6e9e req-31cb156e-e2bd-41ac-8766-e09eeeaacf82 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:18.431457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-2a42e29e-3fbc-422a-9ad1-25f9578d6e9e req-31cb156e-e2bd-41ac-8766-e09eeeaacf82 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:18.460902 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-2a42e29e-3fbc-422a-9ad1-25f9578d6e9e req-31cb156e-e2bd-41ac-8766-e09eeeaacf82 service nova] Creating event network-changed:eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f for instance 01a19441-142f-41aa-9253-c5b50c929e93 on hv-cinder-zed-3 Oct 28 11:04:18.467100 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-2a42e29e-3fbc-422a-9ad1-25f9578d6e9e req-31cb156e-e2bd-41ac-8766-e09eeeaacf82 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056749 Oct 28 11:04:18.476355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 179/370] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:18.611365 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:18.611846 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:18.617840 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:18.618657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:18.619393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:18.620457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:18.621122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:18.621749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:18.893957 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:18.909941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2bad5dcc-92d4-4267-b094-bb8429b8d704 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1454 microversion: 2.1 time: 0.300168 Oct 28 11:04:18.911723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 192/371] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:18 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1454 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:18.939293 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:18.940906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:18.949866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:18.950614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:18.951203 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:18.952057 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:18.952665 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:18.953271 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:19.173346 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:19.185126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e6eb4b2e-1576-4aa3-ae79-ff8b2fb68c9c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1454 microversion: 2.1 time: 0.253104 Oct 28 11:04:19.198715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 180/372] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:18 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1454 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:21.969918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-bd75747a-f516-47dd-9636-bb41484303b8 req-d040abac-bdcf-420e-86d6-b5b1643810fa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "917e11e6-71de-4156-bffc-7a5a81135795", "tag": "4fc47c92-3eb6-4a2b-85be-a8582c825e05"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:21.981248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:21.983779 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-bd75747a-f516-47dd-9636-bb41484303b8 req-d040abac-bdcf-420e-86d6-b5b1643810fa service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:21.984793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-bd75747a-f516-47dd-9636-bb41484303b8 req-d040abac-bdcf-420e-86d6-b5b1643810fa service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:21.985458 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-bd75747a-f516-47dd-9636-bb41484303b8 req-d040abac-bdcf-420e-86d6-b5b1643810fa service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:22.002833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-bd75747a-f516-47dd-9636-bb41484303b8 req-d040abac-bdcf-420e-86d6-b5b1643810fa service nova] Creating event network-changed:4fc47c92-3eb6-4a2b-85be-a8582c825e05 for instance 917e11e6-71de-4156-bffc-7a5a81135795 on hv-cinder-zed-3 Oct 28 11:04:22.011308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-bd75747a-f516-47dd-9636-bb41484303b8 req-d040abac-bdcf-420e-86d6-b5b1643810fa service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044935 Oct 28 11:04:22.012600 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 193/373] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:24.760247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-857725590", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "networks": [{"uuid": "125d4da4-285b-4dff-aab9-ade58d34df72"}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:24.834074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] validate_networks() for [('125d4da4-285b-4dff-aab9-ade58d34df72', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:04:25.560636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:04:25.561061 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:04:25.561531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:04:25.562449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:04:25.562858 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:04:25.572704 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:25.573289 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:25.573802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:25.574810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:25.575920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:25.576423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:25.595304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Getting quotas for project fc436cb987344c159e565353054e9539. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:04:25.599691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Getting quotas for user 8279ddc694824ebc8185f50fafbeec76 and project fc436cb987344c159e565353054e9539. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:04:25.611382 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:04:25.613797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:04:25.664835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-666ad45a-eb3a-4362-94fc-e5ad036dfb16 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.910142 Oct 28 11:04:25.665995 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 181/374] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:04:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 912 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:04:25.677499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5ac206af-0cfc-4d2a-95ca-70a62abf9e80 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:25.678783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5ac206af-0cfc-4d2a-95ca-70a62abf9e80 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:25.763737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5ac206af-0cfc-4d2a-95ca-70a62abf9e80 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:25.779971 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5ac206af-0cfc-4d2a-95ca-70a62abf9e80 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1213 microversion: 2.1 time: 0.106638 Oct 28 11:04:25.780721 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 194/375] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:25 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1213 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:26.804963 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:26.805802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:26.814222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:26.815116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:26.815916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:26.816872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:26.818292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:26.834777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:26.951578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:26.965679 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-666ada04-5ead-40c5-aa42-66e1fc303a6e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1265 microversion: 2.1 time: 0.165849 Oct 28 11:04:26.969880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 182/376] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:26 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1265 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:27.993054 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:27.995796 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:28.011301 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:28.012213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:28.013145 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:28.014344 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:28.015070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:28.016628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:28.153311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:28.164701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-80a9c4e2-39c7-4397-af58-28f2dd2525ad tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1283 microversion: 2.1 time: 0.179517 Oct 28 11:04:28.165270 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 195/377] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:27 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1283 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:29.192982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:29.196584 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:29.219154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:29.219910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:29.222007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:29.224478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:29.225120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:29.228623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:29.258845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:29.509411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:29.518565 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4856036c-711c-4d81-a4d2-6298b9bf8b0b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.333170 Oct 28 11:04:29.522278 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 183/378] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:29 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:30.023564 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-c0c6b033-2cd6-46d7-a033-ba0d7d615096 req-2aa02187-bf85-49b7-b668-28a861f60af6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "74805231-5f7b-4de3-8151-06083c3c8bbd", "tag": "e06a6103-d1c7-40a3-a1c7-18b7e291548e"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:30.072960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c0c6b033-2cd6-46d7-a033-ba0d7d615096 req-2aa02187-bf85-49b7-b668-28a861f60af6 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:30.073475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c0c6b033-2cd6-46d7-a033-ba0d7d615096 req-2aa02187-bf85-49b7-b668-28a861f60af6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:30.073928 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c0c6b033-2cd6-46d7-a033-ba0d7d615096 req-2aa02187-bf85-49b7-b668-28a861f60af6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:30.097018 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-c0c6b033-2cd6-46d7-a033-ba0d7d615096 req-2aa02187-bf85-49b7-b668-28a861f60af6 service nova] Creating event network-changed:e06a6103-d1c7-40a3-a1c7-18b7e291548e for instance 74805231-5f7b-4de3-8151-06083c3c8bbd on hv-cinder-zed-3 Oct 28 11:04:30.102427 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-c0c6b033-2cd6-46d7-a033-ba0d7d615096 req-2aa02187-bf85-49b7-b668-28a861f60af6 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096217 Oct 28 11:04:30.103495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 196/379] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:30.549000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:30.549879 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:30.558970 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:30.559878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:30.560372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:30.562634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:30.563219 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:30.563754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:30.857653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:30.869465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7d612074-52a8-4fbc-b3a6-748214a4b04d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.324411 Oct 28 11:04:30.870365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 184/380] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:30 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:31.895124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:31.895743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:31.907073 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:31.908986 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:31.909596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:31.911477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:31.914176 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:31.915374 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:32.527704 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:32.536367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fb792104-0192-4499-a8c9-3c9bb9e4b983 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.645302 Oct 28 11:04:32.537185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 197/381] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:31 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:33.560559 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:33.561696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:33.568992 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:33.569760 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:33.570407 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:33.571439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:33.572048 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:33.572754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:33.741180 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:33.752092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-742cb9e1-420f-4355-a08d-94a3e518e7d8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.195875 Oct 28 11:04:33.753537 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 185/382] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:33 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:34.776992 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:34.784134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:34.792275 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:34.793126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:34.793782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:34.794875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:34.795573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:34.796509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:34.988618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:34.998146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1ac90c29-1e7d-4246-9964-8f313281ac03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.225397 Oct 28 11:04:34.999284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 198/383] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:34 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:36.024796 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:36.025764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:36.032729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:36.035661 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:36.037536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:36.038639 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:36.039270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:36.040242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:36.040947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:36.041761 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:36.247442 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:36.255934 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ab396fa1-684d-459a-9882-e383a9a4a5a5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.236617 Oct 28 11:04:36.271568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 186/384] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:36 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:37.283110 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:37.284429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:37.290855 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:37.292908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:37.293816 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:37.294504 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:37.295446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:37.296090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:37.299492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:37.305584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:37.451403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:37.460188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ff61a8db-bc6c-47d1-bb28-355bc49198da tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.183377 Oct 28 11:04:37.461018 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 199/385] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:37 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:38.485851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:38.490459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:38.500715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:38.508939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:38.509474 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:38.510251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:38.510720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:38.511140 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:38.679134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:38.687260 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-63f847e7-d843-422b-8b82-7dedfe3b830f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.207485 Oct 28 11:04:38.689421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 187/386] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:38 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:39.712559 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:39.713178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:39.719511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:39.720260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:39.720926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:39.721849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:39.722530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:39.723983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:39.873538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:39.882243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-465d82fc-aeed-4c16-bad5-7114dfe95ddd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.174697 Oct 28 11:04:39.888973 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 200/387] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:39 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:40.905124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:40.905835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:40.912453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:40.913161 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:40.913802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:40.914578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:40.915125 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:40.915542 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:41.053190 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:41.061090 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2def7c8e-8da2-449c-9947-f9bda62627fb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.159927 Oct 28 11:04:41.061652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 188/388] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:40 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:42.085134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:42.086414 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:42.093428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:42.094183 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:42.094781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:42.095644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:42.096098 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:42.096517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:42.235359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:42.244658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1cf8e6b0-ee12-4d0f-a99d-7cd66ec8b1e1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.164015 Oct 28 11:04:42.254810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 201/389] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:42 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:43.289550 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:43.290207 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:43.296920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:43.297609 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:43.298069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:43.299085 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:43.299551 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:43.300057 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:43.469242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:43.479139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-734d6877-8282-4d05-a94b-1f57ad212505 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.201493 Oct 28 11:04:43.480575 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 189/390] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:43 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:44.046295 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f9823620-53af-4f14-8bd1-909d058c4986 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Action: 'create', calling method: .version_select at 0x7f42faa29820>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1518528129", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDo5I8xFBsIDPhDve+Hrzs0A8vt9Q02nsjqXatq7ooksadH6tPLVNnxWet8FKbZ2urPuC7YmAnnPeCquzhNOfE9eMHN8OmE/5txzUYNF5ez/BifoGMc0MMmKWtmDmG4T/A=="}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:44.061265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-f9823620-53af-4f14-8bd1-909d058c4986 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Getting quotas for project 27b47aa9a61b4898a9af0062896cc3da. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:04:44.065000 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-f9823620-53af-4f14-8bd1-909d058c4986 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Getting quotas for user 859315e3352843fd801ddc6568729e56 and project 27b47aa9a61b4898a9af0062896cc3da. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:04:44.077320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR oslo.messaging._drivers.impl_rabbit [None req-f9823620-53af-4f14-8bd1-909d058c4986 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [b925a8e9-20ed-4e25-9474-778da0105f3b] AMQP server on 10.222.0.46:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 28 11:04:44.502538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:44.503176 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:44.509525 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:44.510022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:44.510455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:44.511126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:44.511571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:44.511985 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:44.646730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:44.654772 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7c4f849c-c197-45c6-8086-01dc0da45d7a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.155955 Oct 28 11:04:44.655331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 190/391] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:44 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:45.084628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO oslo.messaging._drivers.impl_rabbit [None req-f9823620-53af-4f14-8bd1-909d058c4986 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [b925a8e9-20ed-4e25-9474-778da0105f3b] Reconnected to AMQP server on 10.222.0.46:5672 via [amqp] client with port 60336. Oct 28 11:04:45.098182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f9823620-53af-4f14-8bd1-909d058c4986 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 1.055824 Oct 28 11:04:45.098781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 202/392] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:04:44 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 1057 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:04:45.199268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-103205068", "imageRef": "c8877458-0c5d-4a06-a2f4-438bf0988ba0", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1518528129", "networks": [{"uuid": "8e4af541-0222-4578-8070-e116020f3217"}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:45.250211 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] validate_networks() for [('8e4af541-0222-4578-8070-e116020f3217', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:04:45.540934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-8682bce4-b771-43bd-80dc-5f90afb5f6d4 req-ce7730ff-24c4-4b4a-8c67-57c1d5e400ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74805231-5f7b-4de3-8151-06083c3c8bbd", "name": "network-vif-plugged", "status": "completed", "tag": "e06a6103-d1c7-40a3-a1c7-18b7e291548e"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:45.562934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8682bce4-b771-43bd-80dc-5f90afb5f6d4 req-ce7730ff-24c4-4b4a-8c67-57c1d5e400ef service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:45.563416 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8682bce4-b771-43bd-80dc-5f90afb5f6d4 req-ce7730ff-24c4-4b4a-8c67-57c1d5e400ef service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:45.564029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8682bce4-b771-43bd-80dc-5f90afb5f6d4 req-ce7730ff-24c4-4b4a-8c67-57c1d5e400ef service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:45.575544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-8682bce4-b771-43bd-80dc-5f90afb5f6d4 req-ce7730ff-24c4-4b4a-8c67-57c1d5e400ef service nova] Creating event network-vif-plugged:e06a6103-d1c7-40a3-a1c7-18b7e291548e for instance 74805231-5f7b-4de3-8151-06083c3c8bbd on hv-cinder-zed-3 Oct 28 11:04:45.580864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-8682bce4-b771-43bd-80dc-5f90afb5f6d4 req-ce7730ff-24c4-4b4a-8c67-57c1d5e400ef service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043915 Oct 28 11:04:45.581638 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 203/393] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:04:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:04:45.631134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:04:45.631562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:04:45.632590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:04:45.633003 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:04:45.633432 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:04:45.642207 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:45.642763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:45.643143 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:45.643859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:45.644498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:45.644924 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:45.662140 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Getting quotas for project 27b47aa9a61b4898a9af0062896cc3da. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:04:45.665991 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Getting quotas for user 859315e3352843fd801ddc6568729e56 and project 27b47aa9a61b4898a9af0062896cc3da. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:04:45.677877 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:04:45.681155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c8877458-0c5d-4a06-a2f4-438bf0988ba0',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:04:45.681681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:45.682613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:45.690713 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:45.691196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:45.691812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:45.692512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:45.692969 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:45.693584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:45.732910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e1d4c578-0244-4a5c-a592-5ce2c55799d2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.537553 Oct 28 11:04:45.734009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 191/394] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:04:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 539 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:04:45.752238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4874435a-ae2f-4c56-9f1b-691e8b53c6d9 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:45.752787 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4874435a-ae2f-4c56-9f1b-691e8b53c6d9 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:45.815297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4874435a-ae2f-4c56-9f1b-691e8b53c6d9 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:45.823519 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4874435a-ae2f-4c56-9f1b-691e8b53c6d9 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1258 microversion: 2.1 time: 0.080366 Oct 28 11:04:45.826059 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 192/395] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:45 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1258 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:45.911175 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:45.920029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8eb18e0a-69fb-4c5c-829b-43bb087d68b8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.244577 Oct 28 11:04:45.920680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 204/396] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:45 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:46.846009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:46.846676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:46.853231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:46.853709 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:46.854174 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:46.854859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:46.855310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:46.855729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:46.921479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:46.929692 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7f1e3947-e456-4b3c-b053-4e3dfd00357f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1310 microversion: 2.1 time: 0.087535 Oct 28 11:04:46.930364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 193/397] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:46 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1310 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:46.943227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:46.943828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:46.950604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:46.951093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:46.951522 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:46.952216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:46.952661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:46.953076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:47.085737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:47.093555 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-72f5fdb6-0d12-4a8a-aa9c-ec3cc9eb6ac5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1313 microversion: 2.1 time: 0.154055 Oct 28 11:04:47.094341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 205/398] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:46 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:47.953887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:47.954538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:47.961479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:47.961953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:47.962486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:47.963162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:47.963616 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:47.964042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:48.028911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:48.036968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ac2ed091-05ed-4562-a549-f3b81aa7cd35 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1318 microversion: 2.1 time: 0.086894 Oct 28 11:04:48.037543 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 194/399] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:47 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1318 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:48.117159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:48.117761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:48.123809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:48.124279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:48.124706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:48.125388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:48.125833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:48.126276 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:48.281590 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:48.289827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cd6c22b5-3fe3-4ee4-9fff-952f96096df6 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1498 microversion: 2.1 time: 0.176498 Oct 28 11:04:48.290450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 206/400] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:48 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1498 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:48.804968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:48.805788 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:48.816684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:48.818104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:48.818943 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:48.820241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:48.821121 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:48.821563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:48.853889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:04:48.960561 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:48.968886 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8f7e9bf2-f837-4d51-9676-6f23a1183b1f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "DELETE /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 204 len: 0 microversion: 2.1 time: 0.165819 Oct 28 11:04:48.969315 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 195/401] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:04:48 2022] DELETE /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:04:48.978828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:48.979472 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:48.992716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:48.993103 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:48.993539 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:48.994290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:48.994748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:48.995166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:49.067640 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:49.068350 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:49.088856 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:49.089640 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:49.090571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:49.091528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:49.092138 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:49.092717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:49.252928 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:49.262289 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-378e6abe-68fd-4ad9-a1ac-13715ac7e76c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.198796 Oct 28 11:04:49.263184 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 196/402] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:49 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:49.299883 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:49.326441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b99c7548-bb34-415b-b763-d431e63fb428 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.351875 Oct 28 11:04:49.327321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 207/403] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:48 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:50.289782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:50.290420 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:50.296820 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:50.297304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:50.297738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:50.298499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:50.298954 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:50.299376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:50.359277 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:50.360172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:50.371684 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:50.372250 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:50.372620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:50.373653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:50.374135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:50.374556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:50.467229 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:50.474934 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0ef58892-427c-4f62-a0a5-430b2681be39 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.189495 Oct 28 11:04:50.475488 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 197/404] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:50 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:50.530226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:50.538025 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-863d3328-9338-4fc0-8d6a-99c1cb3d539f tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.182689 Oct 28 11:04:50.538695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 208/405] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:50 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:50.734458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:50.735207 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:50.741081 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:50.741556 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:50.742005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:50.743213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:50.743701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:50.744107 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:50.767157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e6af0813-2552-4654-bcd9-94d68170449d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/ips" status: 200 len: 152 microversion: 2.20 time: 0.036724 Oct 28 11:04:50.767718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 198/406] 10.222.0.46 () {60 vars in 1289 bytes} [Fri Oct 28 11:04:50 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/ips => generated 152 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 11:04:51.498106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:51.509616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:51.515809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:51.516296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:51.516729 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:51.517429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:51.517883 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:51.518342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:51.561359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:51.562421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:51.572979 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:51.573929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:51.575221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:51.577169 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:51.578739 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:51.580329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:51.664272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:51.672588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0be97ec0-095d-42a2-b6af-bcbb23a0a66e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.178441 Oct 28 11:04:51.673335 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 209/407] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:51 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:51.746900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:51.754689 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8a5c95f5-a82d-472a-ad54-01338682af4d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.197812 Oct 28 11:04:51.755196 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 199/408] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:51 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:52.694643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:52.695308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:52.697241 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=45,nova_api:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:52.702468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:52.702953 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:52.703392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:52.704079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:52.704533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:52.704959 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:52.780781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:52.781381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:52.787331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:52.787803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:52.788377 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:52.788891 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:52.790577 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:52.791030 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:52.807280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:52.852904 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:52.861592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3765d27e-73d5-43b8-852a-943bf9098d7b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.170734 Oct 28 11:04:52.862285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 210/409] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:52 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:52.933374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:52.942566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5768e80e-539e-4e6e-92cc-233b4b0f78f5 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.168164 Oct 28 11:04:52.943141 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 200/410] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:52 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:53.884628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:53.885286 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:53.891531 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:53.892009 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:53.892443 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:53.893138 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:53.893596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:53.894050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:53.966533 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:53.967942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:53.974492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:53.974973 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:53.975404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:53.976093 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:53.976544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:53.976965 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:54.023568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:54.031847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8ec3d4ff-3a62-4359-ae5c-2fb06c12c235 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.151133 Oct 28 11:04:54.032427 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 211/411] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:53 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:54.036993 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:04:54.037461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:54.043616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:54.044095 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:54.044733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:54.045418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:54.045877 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:54.046348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:54.078432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:54.103873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lazy-loading 'flavor' on Instance uuid e4ef9cf5-ad72-4399-a910-9308a243a57a {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:04:54.124112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:54.135365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9d239624-536b-4ae5-8589-f4149a78ed49 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.173159 Oct 28 11:04:54.142900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 201/412] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:53 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:55.057323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:55.058013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:55.064408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:55.065102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:55.065765 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:55.066713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:55.067374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:55.068001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:55.206261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:55.214866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-cad73330-29db-4727-9f96-d9a0eccfe7bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.163738 Oct 28 11:04:55.215437 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 202/413] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:55 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:55.219984 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:55.220586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:55.226207 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:55.226690 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:55.227124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:55.227823 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:55.228259 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:55.229266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:55.361465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:55.368956 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-415863d9-696e-4de7-8dc4-ecb5ff6dbdb0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.152594 Oct 28 11:04:55.370082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 203/414] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:55 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:56.073901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-655167a0-661e-4cf6-9b91-1b5adca74446 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/action" status: 202 len: 0 microversion: 2.20 time: 2.039729 Oct 28 11:04:56.074550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 212/415] 10.222.0.46 () {62 vars in 1319 bytes} [Fri Oct 28 11:04:54 2022] POST /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/action => generated 0 bytes in 2042 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:04:56.082865 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:56.083479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:56.085295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:56.086258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=8 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:56.091923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:56.092968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:56.093117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:56.094019 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:56.094678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:56.095094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:56.232148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:56.238916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:56.239639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:56.240518 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:56.242018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:56.243257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-34737fab-2db0-4354-870a-7d4858990a22 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.164241 Oct 28 11:04:56.243817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 204/416] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:04:56 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:04:56.246678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:56.247159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:56.247591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:56.248279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:56.248731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:56.249149 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:56.387381 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:56.393063 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:56.393678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:56.398271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dda203da-507b-4c88-b054-8e5b3d446c6e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.163786 Oct 28 11:04:56.399601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:56.400100 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:56.400537 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:56.401273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:56.402342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:56.402770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:56.409232 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 213/417] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:56 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:04:56.551915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:56.560410 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c64aa0ac-5984-4c4d-83d6-f63e9b06fa8e tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.171345 Oct 28 11:04:56.560976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 205/418] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:56 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:57.269361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:57.269998 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:57.276151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:57.277204 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:57.277689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:57.278512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:57.279135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:57.279566 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:57.428579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:57.434222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:57.434829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:57.437542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:57.439032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:57.440194 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dc58fce6-db2f-4001-94db-25f7a239f6a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.174749 Oct 28 11:04:57.440748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 214/419] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:04:57 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:04:57.441185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:57.441661 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:57.442152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:57.442837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:57.443284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:57.443730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:57.568764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:57.576439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-51fb7622-44f5-4412-8881-1fdfe065ac3f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.146525 Oct 28 11:04:57.576947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 206/420] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:57 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:57.584271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:57.584882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:57.591485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:57.591956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:57.592384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:57.593055 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:57.593505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:57.593917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:57.612643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:04:57.744163 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:57.751974 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-14c75c36-4c2e-4c3a-9978-cf0807b931d1 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.171499 Oct 28 11:04:57.752528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 215/421] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:57 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:58.465794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:58.466420 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:58.473089 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:58.473571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:58.474021 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:58.474716 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:58.475358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:58.475776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:58.599383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:58.600023 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:58.603015 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:58.606569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:58.607063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:58.607497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:58.608192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:58.608647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:58.609072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:58.611966 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:58.613406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:58.614583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e96d446f-547a-4de2-a149-52a0804fcdc0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.153059 Oct 28 11:04:58.615147 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 207/422] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:04:58 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:04:58.753395 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:58.763695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0e3f64b5-b203-4ea8-b906-7aae30d26ee2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.168026 Oct 28 11:04:58.764278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 216/423] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:58 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:58.775097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:58.775730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:58.783890 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:58.784375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:58.784808 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:58.785517 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:58.785991 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:58.786423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:58.917071 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:58.925247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-44ef99c3-3cff-4a8e-9145-4f76556da6a8 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.154112 Oct 28 11:04:58.925804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 208/424] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:58 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:59.640267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:59.640888 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:59.647332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:59.648113 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:59.648730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:59.649625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:59.650288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:59.650891 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:59.779804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:59.788028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:59.789480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:59.790486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:59.791001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-af5d3421-64c7-48b9-8437-ef2ee416965c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.154388 Oct 28 11:04:59.791098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:59.791499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 217/425] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:04:59 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:04:59.798109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:59.798590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:59.799026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:59.799716 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:59.800167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:59.800588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:59.924370 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:04:59.932067 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2915a9d9-818e-41e5-9e79-810d8da0dbce tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.145322 Oct 28 11:04:59.932629 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 209/426] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:59 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:04:59.947936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:04:59.949107 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:04:59.954951 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:59.955430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:59.955923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:04:59.956608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:04:59.957057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:04:59.957477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:00.084686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:00.092795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dcd511cb-05b6-4b0e-9004-cefdb0cbc5ca tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.148699 Oct 28 11:05:00.093361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 218/427] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:04:59 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:00.818544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:00.819114 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:00.825773 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:00.826287 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:00.826723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:00.827423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:00.827875 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:00.828299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:00.954582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:00.955198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:00.961072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:00.961550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:00.962004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:00.962756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:00.962874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:00.963157 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:00.963582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:00.971077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:00.972508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:00.973655 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-89f38956-2c6e-4e50-a6a6-8f7830c8dbaf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.160290 Oct 28 11:05:00.974241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 210/428] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:00 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:01.089688 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:01.097231 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1e641423-372c-46e9-a44d-09c8c20d4969 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.146548 Oct 28 11:05:01.097797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 219/429] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:00 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:01.116552 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:01.117153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:01.123056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:01.123526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:01.123949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:01.124621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:01.125064 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:01.125475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:01.250553 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:01.258458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-af95b881-2feb-4e9e-8868-2359e312c81b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.145736 Oct 28 11:05:01.259014 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 211/430] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:01 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:01.999832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:02.000435 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:02.007255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:02.007742 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:02.008179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:02.008876 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:02.009332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:02.009759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:02.120612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:02.121213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:02.127537 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:02.128023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:02.128459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:02.129150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:02.129604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:02.130051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:02.141244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:02.149377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:02.150862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:02.152609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7a9e3404-3eb7-4063-b6fb-b828e8f5bfe2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.156692 Oct 28 11:05:02.153173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 220/431] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:01 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:02.283046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:02.284060 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:02.293907 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:02.294415 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:02.294848 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:02.295573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:02.296015 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:02.296432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:02.300930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:02.311091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0afea782-b938-4a4b-80ed-1e0b6978d5ae tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.194333 Oct 28 11:05:02.311672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 212/432] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:02 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:02.433524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:02.441417 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b36e161c-4e21-4193-a5d0-f05114685d70 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.162576 Oct 28 11:05:02.442304 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 221/433] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:02 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:03.188268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:03.188875 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:03.196775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:03.200653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:03.201317 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:03.202286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:03.203024 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:03.203461 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:03.334797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:03.335792 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:03.342851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:03.343429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:03.343934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:03.344705 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:03.345160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:03.345640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:03.479658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:03.488235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-66ec7433-a214-4221-9afa-1122f8445f76 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.157301 Oct 28 11:05:03.488921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 222/434] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:03 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:03.493623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:03.494343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:03.501317 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:03.502070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:03.502588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:03.503299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:03.503751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:03.504360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:03.578311 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:03.587873 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:03.589411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:03.590608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0975e495-c119-4d36-89bb-b69d9175e284 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.408044 Oct 28 11:05:03.591166 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 213/435] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:03 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:03.637998 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:03.645605 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-64a89d41-71e8-4fc4-8901-9fc0433fd07d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.155658 Oct 28 11:05:03.646189 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 223/436] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:03 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:04.512423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:04.513062 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:04.519516 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:04.519989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:04.520418 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:04.521097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:04.521552 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:04.521992 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:04.615869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:04.616983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:04.622676 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:04.623157 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:04.623590 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:04.624421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:04.624910 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:04.625335 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:04.646162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:04.654511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e1de3725-159a-41a8-a0fc-a8d497119042 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.146314 Oct 28 11:05:04.655155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 214/437] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:04 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:04.670645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:04.670645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:04.675326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:04.675803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:04.676231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:04.676947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:04.677426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:04.677842 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:04.756612 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:04.765524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:04.767014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:04.768177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-98b615a1-90b5-4a20-8c5f-7ecd7ae0df81 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.156278 Oct 28 11:05:04.768741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 224/438] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:04 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:04.812574 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:04.820545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5c538de3-eaa2-4624-b173-f82dd2c1c40a tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.155956 Oct 28 11:05:04.821118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 215/439] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:04 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:05.676190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:05.676802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:05.683419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:05.683901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:05.684336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:05.685022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:05.685477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:05.685903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:05.794395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:05.795131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:05.801527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:05.802032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:05.802469 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:05.803164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:05.803613 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:05.804034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:05.812875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:05.821752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ec9e2f8e-43ad-41d2-9aa5-086e219ae895 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.149368 Oct 28 11:05:05.822363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 225/440] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:05 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:05.843606 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:05.844213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:05.850673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:05.851160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:05.851597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:05.852290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:05.852746 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:05.853173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:05.934078 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:05.942398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:05.943831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:05.945988 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a0be0cec-b392-4e76-9e40-217378afdc75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.155481 Oct 28 11:05:05.946558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 216/441] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:05 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:05.979537 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:05.987880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-72a381fd-fbb9-4c35-acb9-8587fb25896b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.148028 Oct 28 11:05:05.988439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 226/442] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:05 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:06.845192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:06.845806 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:06.852363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:06.852863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:06.853276 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:06.853959 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:06.854443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:06.854870 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:06.858758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:06.971482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:06.972085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:06.977963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:06.978468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:06.978898 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:06.979586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:06.980073 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:06.980499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:06.982423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:06.990299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e97854e0-8e9a-4c85-a04c-09e970057b97 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.148952 Oct 28 11:05:06.990899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 217/443] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:06 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:07.012558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:07.013172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:07.019817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:07.020581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:07.021119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:07.022108 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:07.022700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:07.023201 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:07.110181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:07.118347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:07.119792 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:07.121568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b8f5a3a2-904b-4266-94c4-b10983625abb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.153358 Oct 28 11:05:07.122354 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 227/444] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:06 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:07.152763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:07.160364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-bdc7bbd5-d73c-477f-bd52-74dddf5f9f3b tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.152277 Oct 28 11:05:07.160925 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 218/445] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:07 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:08.013566 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:08.014220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:08.021210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:08.021696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:08.022173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:08.022859 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:08.023311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:08.023735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:08.146134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:08.147231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:08.147837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:08.154361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5fee0577-eb1b-4572-ad36-4ada807cb821 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.144549 Oct 28 11:05:08.154847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:08.154960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 228/446] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:08 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:08.155393 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:08.155904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:08.156595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:08.157040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:08.157455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:08.183181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:08.183736 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:08.189716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:08.190223 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:08.190658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:08.191337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:08.191785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:08.192205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:08.285527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:08.293678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:08.295150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:08.296308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-acc48771-52c5-47ad-acff-011d11e75303 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.153083 Oct 28 11:05:08.296869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 219/447] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:08 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:08.319582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:08.327982 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0a47d1dd-cd36-4004-b61e-89de7cc84457 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.148571 Oct 28 11:05:08.328548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 229/448] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:08 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:09.177206 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:09.177811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:09.184132 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:09.184601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:09.185023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:09.185697 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:09.186172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:09.186590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:09.315733 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:09.320422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:09.321032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:09.323384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b4f542fe-670b-4167-bb01-81504ad177eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.150012 Oct 28 11:05:09.324562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 220/449] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:09 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:09.327583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:09.328066 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:09.328496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:09.329178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:09.329628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:09.330073 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:09.350179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:09.350790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:09.357356 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:09.357839 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:09.358327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:09.359002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:09.359452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:09.360138 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:09.458696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:09.466837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:09.468288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:09.469448 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3b83950f-fb11-4a1d-8b85-272642aecefe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.152814 Oct 28 11:05:09.470025 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 230/450] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:09 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:09.487615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:09.495178 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8e24e1e8-96b7-4678-abb6-d4c1497d7ff4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.148669 Oct 28 11:05:09.496270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 221/451] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:09 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:10.347871 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:10.348474 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:10.355886 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:10.356368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:10.356798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:10.357489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:10.357964 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:10.358422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:10.495364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:10.495940 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:10.502200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:10.502670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:10.503094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:10.503775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:10.504220 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:10.504671 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:10.549556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:10.557104 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a3686892-741b-4d14-b9e6-b3027e8c79e6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.214018 Oct 28 11:05:10.557885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 231/452] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:10 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:10.570534 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:10.571969 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:10.579084 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:10.579567 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:10.579992 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:10.580820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:10.581271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:10.581687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:10.586066 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:10.641803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:10.649842 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:10.651354 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:10.653083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0517aee0-c062-49da-a393-ea12be641009 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.161702 Oct 28 11:05:10.653639 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 222/453] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:10 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:10.663800 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-10ec3308-f80d-4863-a2b6-468f70f4fd3a req-dc5dce9f-f803-4fb5-a918-08779e48e76e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "818ad5c3-a0f1-437e-9fe3-45242c1f7bd3", "tag": "6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:10.673230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-10ec3308-f80d-4863-a2b6-468f70f4fd3a req-dc5dce9f-f803-4fb5-a918-08779e48e76e service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:10.673699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-10ec3308-f80d-4863-a2b6-468f70f4fd3a req-dc5dce9f-f803-4fb5-a918-08779e48e76e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:10.674167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-10ec3308-f80d-4863-a2b6-468f70f4fd3a req-dc5dce9f-f803-4fb5-a918-08779e48e76e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:10.684911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-10ec3308-f80d-4863-a2b6-468f70f4fd3a req-dc5dce9f-f803-4fb5-a918-08779e48e76e service nova] Creating event network-changed:6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 for instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 on hv-cinder-zed-3 Oct 28 11:05:10.689758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-10ec3308-f80d-4863-a2b6-468f70f4fd3a req-dc5dce9f-f803-4fb5-a918-08779e48e76e service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029553 Oct 28 11:05:10.690556 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 223/454] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:05:10.767572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:10.775145 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cc1853f2-9d2d-49fc-ab0d-8abcb2b4a393 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.208950 Oct 28 11:05:10.775833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 232/455] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:10 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:11.581263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:11.581956 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:11.588328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:11.588808 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:11.589734 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:11.590542 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:11.590989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:11.591406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:11.678898 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:11.679778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:11.686068 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:11.686641 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:11.687148 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:11.687913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:11.688632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:11.689133 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:11.725494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:11.734098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a2c931ea-9f2d-4e00-937d-1526c2ca4792 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.157608 Oct 28 11:05:11.734673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 224/456] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:11 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:11.798533 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:11.799272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:11.805268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:11.805834 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:11.806968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:11.807676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:11.808645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:11.809070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:11.889837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:11.898713 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:11.900476 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:11.902022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e67b14ba-877f-4684-8e1f-1cb53d4ff636 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.227269 Oct 28 11:05:11.903097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 233/457] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:11 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:12.082217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:12.090367 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-136c323b-aeea-4985-ad4b-13325f2f80dd tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1504 microversion: 2.1 time: 0.295639 Oct 28 11:05:12.090937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 225/458] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:11 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1504 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:12.465119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-2f560a5f-b238-40c1-899c-b70945194e4a req-97ad4b0d-ad4b-4d7c-92b8-738909f42a56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74805231-5f7b-4de3-8151-06083c3c8bbd", "name": "network-vif-deleted", "tag": "e06a6103-d1c7-40a3-a1c7-18b7e291548e"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:12.474940 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-2f560a5f-b238-40c1-899c-b70945194e4a req-97ad4b0d-ad4b-4d7c-92b8-738909f42a56 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:12.475484 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-2f560a5f-b238-40c1-899c-b70945194e4a req-97ad4b0d-ad4b-4d7c-92b8-738909f42a56 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:12.475983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-2f560a5f-b238-40c1-899c-b70945194e4a req-97ad4b0d-ad4b-4d7c-92b8-738909f42a56 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:12.489198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-2f560a5f-b238-40c1-899c-b70945194e4a req-97ad4b0d-ad4b-4d7c-92b8-738909f42a56 service nova] Creating event network-vif-deleted:e06a6103-d1c7-40a3-a1c7-18b7e291548e for instance 74805231-5f7b-4de3-8151-06083c3c8bbd on hv-cinder-zed-3 Oct 28 11:05:12.498611 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-2f560a5f-b238-40c1-899c-b70945194e4a req-97ad4b0d-ad4b-4d7c-92b8-738909f42a56 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033563 Oct 28 11:05:12.498611 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 234/459] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:05:12.757510 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:12.758176 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:12.764952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:12.765426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:12.765857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:12.766566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:12.767015 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:12.767437 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:12.892029 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:12.899750 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7eb7e6de-e3ff-4adb-a08c-32dc8b832a72 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.146144 Oct 28 11:05:12.900314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 226/460] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:12 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:12.928419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:12.929105 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:12.935178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:12.935654 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:12.936083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:12.936760 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:12.937210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:12.937630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:13.066982 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:13.075540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:13.076983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:13.078166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9ebbc12e-c808-41cd-8e2f-2390161f5e72 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.153883 Oct 28 11:05:13.078734 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 235/461] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:12 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:13.113773 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:13.114413 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:13.120353 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:13.121623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:13.122305 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:13.123246 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:13.123704 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:13.124131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:13.186834 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:13.194403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c73409dd-4f5e-4487-a9d2-58cb7bcd02c0 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1462 microversion: 2.1 time: 0.084697 Oct 28 11:05:13.194975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 227/462] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:13 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1462 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:13.924051 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:13.924660 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:13.930601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:13.931173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:13.931492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:13.932173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:13.932621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:13.933038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:14.062961 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:14.073322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a0a0c8b6-1054-4c51-b2b0-93378170c728 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.153442 Oct 28 11:05:14.073893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 236/463] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:13 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:14.103606 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:14.104130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:14.111276 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:14.111749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:14.112175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:14.112859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:14.113334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:14.113749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:14.218770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:14.219497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:14.225467 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:14.226059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:14.226501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:14.227187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:14.227641 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:14.228063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:14.240901 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:14.249231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:14.250727 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:14.251920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8f73b0de-efed-456d-a766-772d180e9cc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.152627 Oct 28 11:05:14.252484 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 228/464] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:14 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:14.290198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:14.297794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e0647e92-a9ba-40ea-85a6-6d744dce8bde tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1462 microversion: 2.1 time: 0.083555 Oct 28 11:05:14.298385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 237/465] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:14 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1462 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:15.096523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:15.097360 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:15.104497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:15.105603 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:15.105939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:15.106655 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:15.107343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:15.107775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:15.237763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:15.246232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3db5fcf1-5c2e-468f-94a4-d9332026ca05 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.153363 Oct 28 11:05:15.247017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 229/466] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:15 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:15.279430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:15.280020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:15.286424 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:15.286906 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:15.287341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:15.288024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:15.288476 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:15.288926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:15.320499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:15.321136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:15.327012 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:15.327491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:15.327968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:15.330030 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:15.330488 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:15.331116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:15.363879 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:15.364358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:15.364797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:15.365235 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lazy-loading 'fault' on Instance uuid 74805231-5f7b-4de3-8151-06083c3c8bbd {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:05:15.411117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:15.419301 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e7402048-76de-4f3f-8ea0-fa0fc7884e2d tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 200 len: 1303 microversion: 2.1 time: 0.102596 Oct 28 11:05:15.419867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 230/467] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:15 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 1303 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:15.423689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:15.431624 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:15.433124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:15.434297 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d5940a42-f473-4900-b92c-ff2b225363e0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.159523 Oct 28 11:05:15.434845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 238/468] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:15 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:16.269181 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:16.270555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:16.280806 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:16.281467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:16.282091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:16.282941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:16.283545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:16.284115 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:16.442426 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:16.443176 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 74805231-5f7b-4de3-8151-06083c3c8bbd] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:16.450001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:16.450588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:16.451096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:16.451859 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:16.452387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:16.452816 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:16.466382 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:16.468191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] HTTP exception thrown: Instance 74805231-5f7b-4de3-8151-06083c3c8bbd could not be found. Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Request method failure captured: Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd HTTP/1.1 Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7c1bP8th_BAqVD4Ta52PA5-nksgBgFNokA0w-GTnoO8Wja4przTrPfH_OBHG8YK2WFCY6htr1kPfrPYlniZdR3I5qZybjHXuiYq7kPjSBXCL5tAxtTM1ogFGFLUJ6rG2DjARQ3-klVm83jpAP-2T3SM0fBENaUSUbI1YtCMnlkKda4 Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: fc436cb987344c159e565353054e9539 Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesActionsTest-1975314171 Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:05:16.470220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/fc436cb987344c159e565353054e9539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_fc436cb987344c159e565353054e9539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/fc436cb987344c159e565353054e9539"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/fc436cb987344c159e565353054e9539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesActionsTest-1975314171 Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: fc436cb987344c159e565353054e9539 Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesActionsTest-1975314171 Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesActionsTest-1975314171-project Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 8279ddc694824ebc8185f50fafbeec76 Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesActionsTest-1975314171-project Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 74805231-5f7b-4de3-8151-06083c3c8bbd could not be found. Oct 28 11:05:16.471921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '74805231-5f7b-4de3-8151-06083c3c8bbd'} Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 74805231-5f7b-4de3-8151-06083c3c8bbd could not be found. Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:05:16.472679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 74805231-5f7b-4de3-8151-06083c3c8bbd could not be found. Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 74805231-5f7b-4de3-8151-06083c3c8bbd could not be found. Oct 28 11:05:16.474057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:05:16.475181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Returning 404 to user: Instance 74805231-5f7b-4de3-8151-06083c3c8bbd could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:05:16.475181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2bc997d3-c3cb-43f1-a455-a79e7a049835 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd" status: 404 len: 111 microversion: 2.1 time: 0.033021 Oct 28 11:05:16.475181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 239/469] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:16 2022] GET /compute/v2.1/servers/74805231-5f7b-4de3-8151-06083c3c8bbd => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:05:16.477623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:16.478528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:16.484307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:16.485298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:16.485509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:16.486224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:16.486673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:16.487086 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:16.491076 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3944379f-f21d-4b1f-984e-bca23cb75231 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.223467 Oct 28 11:05:16.507206 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 231/470] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:16 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:16.619335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1828572387", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "networks": [{"uuid": "125d4da4-285b-4dff-aab9-ade58d34df72"}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:16.644586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:16.654197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:16.655970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:16.657483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6512239f-57e3-4226-8cba-05c1f8940874 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.184467 Oct 28 11:05:16.658589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 240/471] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:16 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:16.689156 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] validate_networks() for [('125d4da4-285b-4dff-aab9-ade58d34df72', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:05:17.044735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:05:17.045146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:05:17.045605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:05:17.046029 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:05:17.046433 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:05:17.055165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:17.055634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:17.056059 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:17.056773 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:17.057217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:17.057630 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:17.074954 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Getting quotas for project fc436cb987344c159e565353054e9539. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:05:17.078911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Getting quotas for user 8279ddc694824ebc8185f50fafbeec76 and project fc436cb987344c159e565353054e9539. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:05:17.089673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:05:17.091968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:05:17.136920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b31f54e2-e6cb-48bd-ad4e-91e2551f2f14 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.522124 Oct 28 11:05:17.137546 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 232/472] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:05:16 2022] POST /compute/v2.1/servers => generated 380 bytes in 524 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 11:05:17.148693 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-58a6564e-a3b1-47c5-a2e4-750f6618c291 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:17.149307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-58a6564e-a3b1-47c5-a2e4-750f6618c291 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:17.202202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-58a6564e-a3b1-47c5-a2e4-750f6618c291 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:17.209443 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-58a6564e-a3b1-47c5-a2e4-750f6618c291 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1214 microversion: 2.1 time: 0.064476 Oct 28 11:05:17.210021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 241/473] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:17 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1214 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:17.514514 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:17.515344 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:17.521298 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:17.521775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:17.522249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:17.522933 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:17.523413 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:17.523831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:17.660416 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:17.669409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a9b1f1d5-e143-49b0-ad7a-fcf432b25ed6 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.159350 Oct 28 11:05:17.670551 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 233/474] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:17 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:17.691192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:17.691794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:17.697708 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:17.698235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:17.698706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:17.699578 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:17.700023 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:17.700439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:17.724404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-1bccbc10-bb3a-4283-b5b1-7443ef659a64 req-ff336a46-87b2-4371-b3d7-bdec307f6b55 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "818ad5c3-a0f1-437e-9fe3-45242c1f7bd3", "name": "network-vif-plugged", "status": "completed", "tag": "6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:17.735530 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1bccbc10-bb3a-4283-b5b1-7443ef659a64 req-ff336a46-87b2-4371-b3d7-bdec307f6b55 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:17.736399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1bccbc10-bb3a-4283-b5b1-7443ef659a64 req-ff336a46-87b2-4371-b3d7-bdec307f6b55 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:17.737119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-1bccbc10-bb3a-4283-b5b1-7443ef659a64 req-ff336a46-87b2-4371-b3d7-bdec307f6b55 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:17.763981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-1bccbc10-bb3a-4283-b5b1-7443ef659a64 req-ff336a46-87b2-4371-b3d7-bdec307f6b55 service nova] Creating event network-vif-plugged:6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 for instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 on hv-cinder-zed-3 Oct 28 11:05:17.770073 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-1bccbc10-bb3a-4283-b5b1-7443ef659a64 req-ff336a46-87b2-4371-b3d7-bdec307f6b55 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049005 Oct 28 11:05:17.771041 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 234/475] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:05:17.873191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:17.882738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:17.884433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:17.885742 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-74b37886-b295-4ee8-89d1-71ebdf4924c8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.198462 Oct 28 11:05:17.887003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 242/476] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:17 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:18.232564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:18.233197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:18.239492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:18.239963 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:18.240392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:18.241068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:18.241514 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:18.241930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:18.273373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:18.325330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:18.332982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-98beb809-9b01-4252-bab1-8fa8a0fa4ac2 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1266 microversion: 2.1 time: 0.105216 Oct 28 11:05:18.333533 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 235/477] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:18 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1266 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:18.693639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:18.694321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:18.700854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:18.701324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:18.701752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:18.702484 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:18.702935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:18.703350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:18.831260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:18.839945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6abe7e20-94af-448c-9766-bc78895302bd tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1358 microversion: 2.1 time: 0.149452 Oct 28 11:05:18.840513 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 243/478] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:18 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1358 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:18.912188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:18.912825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:18.915109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:18.920732 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:18.921211 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:18.921639 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:18.922361 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:18.922798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:18.923217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:19.050905 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:19.058948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:19.060405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:19.061566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2a782e15-bbc6-43f0-b75a-f4a4a64e3fcf tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.153330 Oct 28 11:05:19.062158 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 236/479] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:18 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:19.360278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:19.360278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:19.365947 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:19.366483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:19.367055 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:19.367760 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:19.368217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:19.368641 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:19.435549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:19.443251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e4c1df91-7a6d-41ef-a53a-042795147673 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1284 microversion: 2.1 time: 0.089045 Oct 28 11:05:19.443804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 244/480] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:19 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1284 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:19.862686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:19.863223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:19.870213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:19.870738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:19.871173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:19.871869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:19.872326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:19.873068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.005033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.013467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-050c77c8-c018-4064-a11b-388693ac489c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1547 microversion: 2.1 time: 0.154654 Oct 28 11:05:20.014275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 237/481] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:19 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1547 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:20.036388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:20.036994 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:20.043664 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.044272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.045027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.045714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.046324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.047866 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.086696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:20.087297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:20.093605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.094103 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.094530 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.095208 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.095652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.096116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.185832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.194883 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c3285557-4048-4721-a250-60390b37caed tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1547 microversion: 2.1 time: 0.162250 Oct 28 11:05:20.195591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 245/482] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:20 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1547 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:20.219342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:20.224770 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.225632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Searching by: {'deleted': False, 'project_id': '27b47aa9a61b4898a9af0062896cc3da'} {{(pid=120543) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 28 11:05:20.232562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.233037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.233462 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.234216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.234696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.235474 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.295480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.303558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.304982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.306164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ea378b9c-9a46-4b77-87dc-57f55d3a3dc1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.223250 Oct 28 11:05:20.310015 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 238/483] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:20 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:20.332973 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.multi_cell_list [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120543) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 28 11:05:20.431953 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a32690dc-5a64-42c5-9942-7448a46cf261 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.218208 Oct 28 11:05:20.432450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 246/484] 10.222.0.46 () {58 vars in 1125 bytes} [Fri Oct 28 11:05:20 2022] GET /compute/v2.1/servers => generated 342 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:05:20.443990 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:20.444592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:20.450841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.451319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.451748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.452435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.452882 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.453302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.467765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:20.468981 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:20.476359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.477460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.477989 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.478759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.479576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.480237 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.613867 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.622415 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e122aef9-9b46-4be7-bb13-31eb86b91d9c tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.158953 Oct 28 11:05:20.623075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 247/485] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:20 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:20.646827 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:20.654478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d3d5609e-9641-4596-a0f9-3c70d7688cbc tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1547 microversion: 2.1 time: 0.214238 Oct 28 11:05:20.655045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 239/486] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:20 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1547 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:20.959049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-8b712e5a-fcdb-4180-a6f7-ee0570f476f0 req-57219981-c01d-4f7e-ac3e-b3a53001744d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5699f2fa-7608-41d8-98e5-a73e380e5a90", "tag": "e314f892-b3bf-4825-b0bf-a9151de3c38b"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:20.969323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8b712e5a-fcdb-4180-a6f7-ee0570f476f0 req-57219981-c01d-4f7e-ac3e-b3a53001744d service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:20.980221 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8b712e5a-fcdb-4180-a6f7-ee0570f476f0 req-57219981-c01d-4f7e-ac3e-b3a53001744d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:20.981674 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8b712e5a-fcdb-4180-a6f7-ee0570f476f0 req-57219981-c01d-4f7e-ac3e-b3a53001744d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:20.995178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-8b712e5a-fcdb-4180-a6f7-ee0570f476f0 req-57219981-c01d-4f7e-ac3e-b3a53001744d service nova] Creating event network-changed:e314f892-b3bf-4825-b0bf-a9151de3c38b for instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 on hv-cinder-zed-3 Oct 28 11:05:21.015533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-8b712e5a-fcdb-4180-a6f7-ee0570f476f0 req-57219981-c01d-4f7e-ac3e-b3a53001744d service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060066 Oct 28 11:05:21.016426 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 248/487] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:05:21.332250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:21.332857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:21.339691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:21.340165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:21.340655 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:21.341332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:21.341781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:21.342236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:21.540471 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:21.550786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:21.552234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:21.553398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dce181f4-958b-4cb7-8f98-d6aea22d3e18 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.225140 Oct 28 11:05:21.553948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 240/488] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:21 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:21.646505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:21.647195 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:21.653811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:21.654313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:21.654749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:21.655430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:21.655881 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:21.656304 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:21.782165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:21.789673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cf413ff6-eb25-477f-88b0-deb40eb87378 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.147397 Oct 28 11:05:21.790271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 249/489] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:21 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:22.247603 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7441ee77-9c3f-487a-9d51-7462f1919efa"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:22.250013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:22.256664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:22.257199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:22.257630 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:22.258592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:22.259045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:22.259597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:22.578589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:22.579196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:22.585021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:22.585492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:22.585925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:22.586637 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:22.587084 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:22.587527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:22.716045 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:22.724078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:22.725526 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:22.726718 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b9f645f6-39b8-4781-8c0e-0d4bf41927e1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.151990 Oct 28 11:05:22.727272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 250/490] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:22 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:22.812776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:22.813408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:22.815354 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=84 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:22.820829 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:22.821307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:22.821737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:22.822441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:22.822895 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:22.823311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:22.841081 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:22.949314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:22.957675 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6e258b03-ae75-478a-a343-80be6a4b0258 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.148669 Oct 28 11:05:22.958272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 251/491] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:22 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:23.752646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:23.753282 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:23.759206 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:23.759675 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:23.760097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:23.760765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:23.761209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:23.761619 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:23.888750 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:23.896789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:23.898288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:23.899421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-13483e38-8dd1-4d79-9de7-76b32648e5f1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.150671 Oct 28 11:05:23.899976 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 252/492] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:23 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:23.980021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:23.980625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:23.987857 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:23.988343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:23.988780 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:23.989467 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:23.989921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:23.990374 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:24.120923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:24.129077 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-bd02107d-c118-45d4-9a26-28497c078325 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.152711 Oct 28 11:05:24.129827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 253/493] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:23 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:24.926069 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:24.926845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:24.934025 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:24.935047 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:24.935485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:24.936382 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:24.936961 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:24.937396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:25.006363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-46a1eaab-7dd0-4671-bb23-b9caad05c34f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.763018 Oct 28 11:05:25.007254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 241/494] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:05:22 2022] POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments => generated 194 bytes in 2765 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:05:25.151770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:25.153432 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:25.160309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:25.161030 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:25.161637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:25.162359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:25.162957 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:25.163531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:25.197581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:25.209159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:25.211676 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:25.212973 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-05d52dbe-47a3-4af5-9ebc-42cada3c8af5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.291173 Oct 28 11:05:25.213632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 254/495] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:24 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:25.375121 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:25.383003 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0c5d3fdf-0a0a-4c8c-8c2e-03ee585f3fcb tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.235268 Oct 28 11:05:25.383652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 242/496] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:25 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:26.239923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:26.240521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:26.246503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:26.247548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:26.247978 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:26.248654 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:26.249098 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:26.249512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:26.378057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:26.386102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:26.387555 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:26.388703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8d25920d-8e8e-4701-8eaf-ef0ab586929e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.152766 Oct 28 11:05:26.389261 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 255/497] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:26 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:26.405726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:26.407228 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:26.412937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:26.413421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:26.413886 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:26.414635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:26.415084 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:26.415504 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:26.542645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:26.552044 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-edfc51c8-fe41-43eb-8e45-984f81c61b42 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.150085 Oct 28 11:05:26.553134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 243/498] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:26 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:27.413697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:27.414270 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:27.416113 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:27.421337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:27.421815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:27.422278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:27.422962 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:27.423416 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:27.423844 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:27.550558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:27.558573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:27.560020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:27.561174 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d96ec6a6-eeae-4d72-8714-e760a1c206e2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.151442 Oct 28 11:05:27.561738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 256/499] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:27 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:27.575452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:27.576061 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:27.578449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:27.579292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:27.585239 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:27.585712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:27.586168 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:27.586851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:27.587295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:27.587711 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:27.716388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:27.723891 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-312617f5-9172-40d2-ba0c-69e79533b344 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.152374 Oct 28 11:05:27.724453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 244/500] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:27 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:28.091552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-d68544b1-2cac-4d25-baa5-3812017e86d1 req-e28ef121-331d-49ad-b215-d47a782e64f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5699f2fa-7608-41d8-98e5-a73e380e5a90", "name": "network-vif-plugged", "status": "completed", "tag": "e314f892-b3bf-4825-b0bf-a9151de3c38b"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:28.101419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d68544b1-2cac-4d25-baa5-3812017e86d1 req-e28ef121-331d-49ad-b215-d47a782e64f3 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:28.101888 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d68544b1-2cac-4d25-baa5-3812017e86d1 req-e28ef121-331d-49ad-b215-d47a782e64f3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:28.102350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d68544b1-2cac-4d25-baa5-3812017e86d1 req-e28ef121-331d-49ad-b215-d47a782e64f3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:28.114134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-d68544b1-2cac-4d25-baa5-3812017e86d1 req-e28ef121-331d-49ad-b215-d47a782e64f3 service nova] Creating event network-vif-plugged:e314f892-b3bf-4825-b0bf-a9151de3c38b for instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 on hv-cinder-zed-3 Oct 28 11:05:28.120041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-d68544b1-2cac-4d25-baa5-3812017e86d1 req-e28ef121-331d-49ad-b215-d47a782e64f3 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032200 Oct 28 11:05:28.120799 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 257/501] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:05:28.586821 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:28.587430 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:28.594364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:28.594960 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:28.595395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:28.596076 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:28.596596 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:28.597013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:28.723558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:28.732605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:28.734105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:28.735518 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ae1cc0c5-e1e6-4e35-8434-140ab27c7ad3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.152597 Oct 28 11:05:28.736335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 245/502] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:28 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:28.751775 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:28.753043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:28.759090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:28.759652 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:28.760085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:28.760752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:28.761195 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:28.761614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:28.889822 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:28.897352 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-586f2d9d-d1f5-4684-81d6-d1c12d113ebf tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.149789 Oct 28 11:05:28.897918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 258/503] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:28 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:29.760664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:29.761262 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:29.768994 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:29.769640 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:29.770104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:29.770874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:29.771323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:29.771741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:29.896951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:29.905102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:29.906577 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:29.907725 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6b4d9a93-4708-4bf1-a78d-dc8640bbe556 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.151029 Oct 28 11:05:29.908289 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 246/504] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:29 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:29.920980 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:29.921589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:29.927434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:29.927914 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:29.928350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:29.929032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:29.929483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:29.929905 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:30.055488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:30.063717 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-63e3b4ad-a3cd-4291-b063-ca97710f4ae4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1314 microversion: 2.1 time: 0.147058 Oct 28 11:05:30.064269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 259/505] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:29 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1314 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:30.152117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-44cceef7-989b-4c99-8170-8e65c2a816d5 req-a0322fcf-b1a1-41e4-a310-32a541d9aea6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ef9cf5-ad72-4399-a910-9308a243a57a", "name": "network-vif-plugged", "status": "completed", "tag": "06fb27d9-2d31-41b8-ad55-60bad7a8ec79"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:30.161718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-44cceef7-989b-4c99-8170-8e65c2a816d5 req-a0322fcf-b1a1-41e4-a310-32a541d9aea6 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:30.162292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-44cceef7-989b-4c99-8170-8e65c2a816d5 req-a0322fcf-b1a1-41e4-a310-32a541d9aea6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:30.162779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-44cceef7-989b-4c99-8170-8e65c2a816d5 req-a0322fcf-b1a1-41e4-a310-32a541d9aea6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:30.174149 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-44cceef7-989b-4c99-8170-8e65c2a816d5 req-a0322fcf-b1a1-41e4-a310-32a541d9aea6 service nova] Creating event network-vif-plugged:06fb27d9-2d31-41b8-ad55-60bad7a8ec79 for instance e4ef9cf5-ad72-4399-a910-9308a243a57a on hv-cinder-zed-3 Oct 28 11:05:30.179734 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-44cceef7-989b-4c99-8170-8e65c2a816d5 req-a0322fcf-b1a1-41e4-a310-32a541d9aea6 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031255 Oct 28 11:05:30.180319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 247/506] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:05:30.933158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:30.933788 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:30.936479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:30.942043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:30.942523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:30.942956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:30.943644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:30.944093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:30.944511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:31.075041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:31.083154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:31.084614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:31.085787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e1b194df-56ee-45c1-986f-5ace5e075d60 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.156542 Oct 28 11:05:31.086375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 260/507] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:30 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:31.086764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:31.087523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:31.094597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:31.095009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:31.095436 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:31.096113 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:31.096558 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:31.097001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:31.454349 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:31.462024 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2a56e602-f2da-4fae-b70e-858f22976ed4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1498 microversion: 2.1 time: 0.379446 Oct 28 11:05:31.462594 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 248/508] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:31 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1498 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:05:31.957013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:31.957519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:31.963338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:31.963810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:31.964243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:31.964922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:31.965377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:31.965797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:31.990276 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:05:32.062353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-84a08e65-e54d-43f4-bf27-74b1b9c2c007 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "DELETE /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 204 len: 0 microversion: 2.1 time: 0.108840 Oct 28 11:05:32.062915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 261/509] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:05:31 2022] DELETE /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:05:32.072153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:32.072758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:32.079497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:32.079971 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:32.080395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:32.081082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:32.081529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:32.081949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:32.112433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:32.113120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:32.119013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:32.119483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:32.119906 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:32.120581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:32.121027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:32.121441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:32.224619 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:32.240306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-37211bba-539f-402f-8e1e-bd38b9732629 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1504 microversion: 2.1 time: 0.173102 Oct 28 11:05:32.242283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 249/510] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:32 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1504 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:32.279841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:32.290318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:32.292038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:32.293490 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1504f20e-b610-4854-9372-2c9e93fa13e9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.185395 Oct 28 11:05:32.306900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 262/511] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:32 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:33.265109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:33.265723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:33.272230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:33.272711 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:33.273146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:33.273835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:33.274322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:33.274743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:33.322903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:33.323545 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:33.329495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:33.329998 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:33.330446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:33.331136 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:33.331596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:33.332024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:33.404877 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:33.412562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3d61169e-ef00-48d6-8b81-a15aba53efff tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1504 microversion: 2.1 time: 0.151367 Oct 28 11:05:33.413134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 250/512] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:33 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1504 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:33.462363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:33.470793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:33.472219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:33.473351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-034a7636-1ffb-4358-afbf-393ddf7b05a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1744 microversion: 2.20 time: 0.154312 Oct 28 11:05:33.473896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 263/513] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:33 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1744 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:34.436647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:34.437962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:34.444707 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:34.445205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:34.445632 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:34.446484 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:34.446941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:34.447365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:34.464215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:34.499521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:34.500107 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:34.506803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:34.507646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:34.508330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:34.509085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:34.514874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:34.515603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:34.600355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:34.615727 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e9b333d9-db58-4dcb-8309-c713706eddd4 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1504 microversion: 2.1 time: 0.183092 Oct 28 11:05:34.617340 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 251/514] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:34 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1504 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:34.714914 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:34.733617 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:34.733617 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:34.733617 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fd866952-8fff-46b6-a4fb-7de5eb2d28d3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.237818 Oct 28 11:05:34.754632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 264/515] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:34 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:34.817412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:34.818146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:34.826968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:34.828009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:34.829117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:34.830396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:34.831335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:34.832604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:35.028333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:35.048196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-b3accacd-1a15-422a-8eb5-4273e655034e req-0708c81a-44b3-4443-8ec0-50ae40095e15 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5699f2fa-7608-41d8-98e5-a73e380e5a90", "name": "network-vif-deleted", "tag": "e314f892-b3bf-4825-b0bf-a9151de3c38b"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:35.056657 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b726842f-3f28-40cf-a1a4-281d76539620 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1593 microversion: 2.1 time: 0.243177 Oct 28 11:05:35.070005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 252/516] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:34 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1593 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:35.071704 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-b3accacd-1a15-422a-8eb5-4273e655034e req-0708c81a-44b3-4443-8ec0-50ae40095e15 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:35.077938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-b3accacd-1a15-422a-8eb5-4273e655034e req-0708c81a-44b3-4443-8ec0-50ae40095e15 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:35.078533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-b3accacd-1a15-422a-8eb5-4273e655034e req-0708c81a-44b3-4443-8ec0-50ae40095e15 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:35.095244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-b3accacd-1a15-422a-8eb5-4273e655034e req-0708c81a-44b3-4443-8ec0-50ae40095e15 service nova] Creating event network-vif-deleted:e314f892-b3bf-4825-b0bf-a9151de3c38b for instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 on hv-cinder-zed-3 Oct 28 11:05:35.100532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-b3accacd-1a15-422a-8eb5-4273e655034e req-0708c81a-44b3-4443-8ec0-50ae40095e15 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057393 Oct 28 11:05:35.101219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 265/517] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:05:35.641269 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:35.642521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:35.651293 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:35.652092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:35.652746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:35.653688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:35.654370 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:35.655003 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:35.690150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:35.691000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:35.691732 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:35.692400 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lazy-loading 'fault' on Instance uuid 5699f2fa-7608-41d8-98e5-a73e380e5a90 {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:05:35.735647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8279ddc694824ebc8185f50fafbeec76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc436cb987344c159e565353054e9539', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:35.744911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7b70eec0-10e7-4204-85e7-8121089106ec tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 200 len: 1304 microversion: 2.1 time: 0.108058 Oct 28 11:05:35.746451 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 253/518] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:35 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 1304 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:35.765908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:35.768726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:35.776790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:35.777528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:35.777969 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:35.778730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:35.779184 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:35.779793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:36.031518 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:36.063290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:36.065469 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:36.070128 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cbc1b49a-4e93-4e75-bc7e-0bbefba6854f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.308398 Oct 28 11:05:36.073274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 266/519] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:35 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:36.770931 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:36.771766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] [instance: 5699f2fa-7608-41d8-98e5-a73e380e5a90] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:36.781684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:36.782203 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:36.783703 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:36.784419 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:36.784874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:36.789634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:36.807855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] HTTP exception thrown: Instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 could not be found. Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Request method failure captured: Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 HTTP/1.1 Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7c1bP8th_BAqVD4Ta52PA5-nksgBgFNokA0w-GTnoO8Wja4przTrPfH_OBHG8YK2WFCY6htr1kPfrPYlniZdR3I5qZybjHXuiYq7kPjSBXCL5tAxtTM1ogFGFLUJ6rG2DjARQ3-klVm83jpAP-2T3SM0fBENaUSUbI1YtCMnlkKda4 Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: fc436cb987344c159e565353054e9539 Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-VolumesActionsTest-1975314171 Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:05:36.810327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/fc436cb987344c159e565353054e9539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_fc436cb987344c159e565353054e9539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/fc436cb987344c159e565353054e9539"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/fc436cb987344c159e565353054e9539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-VolumesActionsTest-1975314171 Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: fc436cb987344c159e565353054e9539 Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-VolumesActionsTest-1975314171 Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-VolumesActionsTest-1975314171-project Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 8279ddc694824ebc8185f50fafbeec76 Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-VolumesActionsTest-1975314171-project Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 could not be found. Oct 28 11:05:36.812087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '5699f2fa-7608-41d8-98e5-a73e380e5a90'} Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 could not be found. Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:05:36.813453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 could not be found. Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 could not be found. Oct 28 11:05:36.815527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:05:36.817731 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] Returning 404 to user: Instance 5699f2fa-7608-41d8-98e5-a73e380e5a90 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:05:36.818772 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d3927d74-0533-425a-bcb5-9a8e79153b03 tempest-VolumesActionsTest-1975314171 tempest-VolumesActionsTest-1975314171-project] 10.222.0.46 "GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90" status: 404 len: 111 microversion: 2.1 time: 0.052151 Oct 28 11:05:36.825688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 254/520] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:36 2022] GET /compute/v2.1/servers/5699f2fa-7608-41d8-98e5-a73e380e5a90 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:05:37.099838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:37.100439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:37.108047 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:37.108528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:37.108957 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:37.109653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:37.110397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:37.110820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:37.384230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:37.387494 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:37.409055 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:37.412387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:37.413839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9770d444-d184-40a8-ae9b-958fe840025b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.319637 Oct 28 11:05:37.416329 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 267/521] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:37 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:37.597946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-03a15459-dcd9-425b-8791-98737e91e868 req-e0dc5dad-88c1-4b5f-9d3c-b9c5ee4e65b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "818ad5c3-a0f1-437e-9fe3-45242c1f7bd3", "tag": "6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:37.622674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-03a15459-dcd9-425b-8791-98737e91e868 req-e0dc5dad-88c1-4b5f-9d3c-b9c5ee4e65b8 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:37.623372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-03a15459-dcd9-425b-8791-98737e91e868 req-e0dc5dad-88c1-4b5f-9d3c-b9c5ee4e65b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:37.624175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-03a15459-dcd9-425b-8791-98737e91e868 req-e0dc5dad-88c1-4b5f-9d3c-b9c5ee4e65b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:37.658890 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-03a15459-dcd9-425b-8791-98737e91e868 req-e0dc5dad-88c1-4b5f-9d3c-b9c5ee4e65b8 service nova] Creating event network-changed:6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 for instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 on hv-cinder-zed-3 Oct 28 11:05:37.664994 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-03a15459-dcd9-425b-8791-98737e91e868 req-e0dc5dad-88c1-4b5f-9d3c-b9c5ee4e65b8 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071071 Oct 28 11:05:37.666273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 255/522] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:05:38.441732 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:38.442497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:38.462522 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:38.463358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:38.464020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:38.464801 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:38.465423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:38.466615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:38.813272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:38.823035 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:38.824634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:38.826826 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2483a4a9-89ed-44b1-be48-5edf003907f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.389927 Oct 28 11:05:38.827627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 268/523] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:38 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:39.165918 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1420206025"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:39.166620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:39.174593 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:39.175086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:39.175597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:39.176294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:39.176746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:39.177182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:39.360900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.network.security_group_api [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Adding security group dea451e8-7965-49ca-9e86-e29c4b0c3b6e to port 6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 Oct 28 11:05:39.735750 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d00c3e16-d02c-4776-b095-7093377bce3d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/action" status: 202 len: 0 microversion: 2.1 time: 0.577171 Oct 28 11:05:39.736394 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 256/524] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:05:39 2022] POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/action => generated 0 bytes in 579 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:05:39.752393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:39.754105 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:39.758034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-28edf564-c13c-4bf9-86eb-d323159baa0c req-4962535d-cc15-4fa6-93fe-4c9880871be1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "818ad5c3-a0f1-437e-9fe3-45242c1f7bd3", "tag": "6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:39.774873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:39.775619 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:39.776232 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:39.777114 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:39.777716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:39.778350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:39.788439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-28edf564-c13c-4bf9-86eb-d323159baa0c req-4962535d-cc15-4fa6-93fe-4c9880871be1 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:39.790376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-28edf564-c13c-4bf9-86eb-d323159baa0c req-4962535d-cc15-4fa6-93fe-4c9880871be1 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:39.790952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-28edf564-c13c-4bf9-86eb-d323159baa0c req-4962535d-cc15-4fa6-93fe-4c9880871be1 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:39.810242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-28edf564-c13c-4bf9-86eb-d323159baa0c req-4962535d-cc15-4fa6-93fe-4c9880871be1 service nova] Creating event network-changed:6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 for instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 on hv-cinder-zed-3 Oct 28 11:05:39.816445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-28edf564-c13c-4bf9-86eb-d323159baa0c req-4962535d-cc15-4fa6-93fe-4c9880871be1 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069060 Oct 28 11:05:39.817306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 257/525] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:05:39.855552 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:39.857156 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:39.864555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:39.865386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:39.866152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:39.867173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:39.867813 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:39.868388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:39.955500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:39.971199 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-62d64d21-96e7-4c06-82ff-d2c73b89ce9c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1640 microversion: 2.1 time: 0.229262 Oct 28 11:05:39.972503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 269/526] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:05:39 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1640 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:05:40.061681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:40.070241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:40.071982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:40.073212 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-00404dbb-72d6-451c-a757-343593734cbe tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.222401 Oct 28 11:05:40.073854 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 258/527] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:39 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:41.103832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:41.104614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:41.111489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:41.111977 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:41.112414 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:41.113121 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:41.113766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:41.114014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:41.282740 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:41.291980 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:41.293403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:41.295686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8489a80d-ffa6-40c8-aa82-892b0aee5067 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.195690 Oct 28 11:05:41.296901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 270/528] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:41 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:42.324160 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:42.324775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:42.331586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:42.332302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:42.332741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:42.333435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:42.333896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:42.334397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:42.488383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:42.498281 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:42.500463 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:42.502126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6248b81b-6714-434c-8d0c-d07befc0bd27 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.181935 Oct 28 11:05:42.518000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 259/529] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:42 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:05:43.529619 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:43.530882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:43.538446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:43.539351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:43.540036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:43.540806 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:43.541478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:43.541910 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:43.701022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:43.710650 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:43.712646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:43.714272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8b492957-e163-4260-8803-515c58fe4052 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.190225 Oct 28 11:05:43.715371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 271/530] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:43 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:44.779579 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:44.781085 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:44.791949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:44.792849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:44.793537 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:44.798255 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:44.799181 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:44.799888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:45.150412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:45.161499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:45.168563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:45.170446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6f6bd610-57c0-4d01-b553-e7b6a94d575a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.433220 Oct 28 11:05:45.175683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 260/531] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:44 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:46.200730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:46.201357 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:46.207434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:46.207921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:46.208367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:46.209063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:46.209556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:46.209926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:46.372683 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:46.382732 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:46.384155 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:46.385316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a28e3793-c6ad-4280-8742-a56bf508fbdc tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1739 microversion: 2.20 time: 0.189097 Oct 28 11:05:46.385863 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 272/532] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:46 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1739 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:47.410249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:47.410939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:47.421052 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:47.421660 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:47.422204 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:47.423117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:47.423641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:47.424273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:47.596672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:47.616768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:47.618682 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:47.620037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2d8c6c23-84d4-4637-a265-8612a9f59661 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1721 microversion: 2.20 time: 0.213783 Oct 28 11:05:47.621346 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 261/533] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:47 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:48.650143 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:48.650760 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:48.657510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:48.658287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:48.658758 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:48.659455 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:48.659908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:48.660332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:48.801186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:48.810004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:48.811444 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:48.812671 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-21fc8521-2f6c-48b3-ac6e-58068e2f7ff0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1721 microversion: 2.20 time: 0.166441 Oct 28 11:05:48.813224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 273/534] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:48 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1721 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:49.838701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:49.839316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:49.841868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=54 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:49.846910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:49.847400 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:49.847837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:49.848544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:49.849001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:49.849427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:50.013193 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:50.038151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:50.039590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:50.040753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fb70941f-6fe0-4333-a1f3-08f2d1a0bea1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1721 microversion: 2.20 time: 0.206079 Oct 28 11:05:50.041310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 262/535] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:49 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1721 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:51.067601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:51.068195 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:51.074469 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.075311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.076014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.076946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.077518 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.078480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.126353 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-cc3dc414-de00-4109-82b4-9111efbb602d req-53d6be02-e074-4850-9d2f-6c88f63504e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ef9cf5-ad72-4399-a910-9308a243a57a", "name": "network-vif-unplugged", "status": "completed", "tag": "06fb27d9-2d31-41b8-ad55-60bad7a8ec79"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:51.148356 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-cc3dc414-de00-4109-82b4-9111efbb602d req-53d6be02-e074-4850-9d2f-6c88f63504e5 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.154642 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-cc3dc414-de00-4109-82b4-9111efbb602d req-53d6be02-e074-4850-9d2f-6c88f63504e5 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.155368 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-cc3dc414-de00-4109-82b4-9111efbb602d req-53d6be02-e074-4850-9d2f-6c88f63504e5 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.180056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.compute.server_external_events [req-cc3dc414-de00-4109-82b4-9111efbb602d req-53d6be02-e074-4850-9d2f-6c88f63504e5 service nova] Unable to find a host for instance e4ef9cf5-ad72-4399-a910-9308a243a57a. Dropping event network-vif-unplugged {{(pid=120544) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 28 11:05:51.182921 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-cc3dc414-de00-4109-82b4-9111efbb602d req-53d6be02-e074-4850-9d2f-6c88f63504e5 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.061308 Oct 28 11:05:51.187936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 263/536] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:05:51 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 67 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 28 11:05:51.271445 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:51.280832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:51.282625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:51.283846 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3e53575a-4089-4710-a694-449769b7e986 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1664 microversion: 2.20 time: 0.220094 Oct 28 11:05:51.284600 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 274/537] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:51 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1664 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:51.309843 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "47acd41a-f217-45c0-88d0-aca5ec5373f9"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:51.312213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:51.318491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.318919 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.319608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.320508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.321137 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.321560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.648664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fe394550-6c48-4a9a-a7aa-57d2be77dfb2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.342669 Oct 28 11:05:51.649326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 264/538] 10.222.0.46 () {62 vars in 1364 bytes} [Fri Oct 28 11:05:51 2022] POST /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments => generated 188 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 11:05:51.779119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:05:51.779713 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:51.786503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.787015 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.787451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.788132 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.788576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.788987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.836602 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120543) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 28 11:05:51.912619 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e6c340ba-44ad-4f88-af1a-1b1227f8d1b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/action" status: 202 len: 0 microversion: 2.20 time: 0.137218 Oct 28 11:05:51.915444 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 275/539] 10.222.0.46 () {62 vars in 1319 bytes} [Fri Oct 28 11:05:51 2022] POST /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/action => generated 0 bytes in 141 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:05:51.924641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:51.925348 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:51.937953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.938481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.938926 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:51.939666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:51.940127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:51.940554 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:52.107821 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:52.121388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:52.124849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:52.126305 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-79d4406e-f6eb-4bca-b49f-57846cc87332 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1750 microversion: 2.20 time: 0.205920 Oct 28 11:05:52.129615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 265/540] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:51 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1750 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:53.156268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:53.158548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:53.160913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=59,nova_api:UPDATE=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:53.176185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:53.180800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:53.181603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:53.183419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:53.184011 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:53.185495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:53.352503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:53.363080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:53.364621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:53.365962 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5589417e-36c2-4a47-8ba8-a301c3f85990 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1752 microversion: 2.20 time: 0.214124 Oct 28 11:05:53.367129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 276/541] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:53 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1752 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:54.393059 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:54.394383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:54.400795 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:54.401482 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:54.402107 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:54.402971 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:54.403438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:54.403844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:54.546574 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:54.558623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:54.562586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:54.564781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-63e4cce1-d07b-4ab7-8d13-2d687edede1e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.175356 Oct 28 11:05:54.579639 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 266/542] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:54 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:55.598934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:55.600110 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:55.626849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:55.627681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:55.628433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:55.629625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:55.630461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:55.631290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:55.877039 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:55.886471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:55.887952 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:55.889160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c0dbc093-a70f-466b-8bcc-3bd41bb8abd3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.299353 Oct 28 11:05:55.889734 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 277/543] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:55 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:56.917572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:56.918405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:56.924694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:56.925180 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:56.925611 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:56.926337 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:56.926789 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:56.927230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:57.099296 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:57.117278 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:57.118781 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:57.120035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1db29c67-7dd6-4e6a-9ea5-efcd14dd6444 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.206785 Oct 28 11:05:57.134006 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 267/544] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:56 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:58.147401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:58.148344 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:58.159324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:58.159924 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:58.160489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:58.161371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:58.162007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:58.162559 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:58.431089 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:58.451386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:58.453773 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:58.455335 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ed19fe0b-ea6e-4313-ae5d-26b6083abed2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.312301 Oct 28 11:05:58.466200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 278/545] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:58 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:05:59.484599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:05:59.486171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:05:59.493254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:59.494010 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:59.494686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:59.495621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:05:59.496151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:05:59.497291 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:05:59.500097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:05:59.646533 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:59.656536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:59.658003 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:05:59.659142 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-cd9ea89d-b3f6-417a-b781-f368f84281c7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.178471 Oct 28 11:05:59.659697 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 268/546] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:05:59 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:00.684668 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:00.685274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:00.691201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:00.691671 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:00.692095 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:00.692768 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:00.693208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:00.693620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:00.828648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:00.839371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:00.841618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:00.842823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f502e35b-a6d1-4c5c-83a8-03743dd3708f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.162105 Oct 28 11:06:00.843730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 279/547] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:00 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:01.872348 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:01.872968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:01.879856 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:01.880334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:01.880759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:01.881457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:01.881894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:01.882359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:02.017094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:02.026498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:02.028680 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:02.030103 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6f627aee-bd9a-433e-93ae-7bcd1f730084 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.161589 Oct 28 11:06:02.035309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 269/548] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:01 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:03.056700 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:03.057706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:03.063839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:03.064314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:03.064741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:03.065425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:03.065866 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:03.066309 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:03.213967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:03.223708 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:03.225316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:03.226501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b0826168-7a56-4daa-b3f0-c3fec29dafe8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.174863 Oct 28 11:06:03.227043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 280/549] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:03 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:04.253009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:04.253619 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:04.259784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:04.260254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:04.260683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:04.261363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:04.261811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:04.262250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:04.401426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:04.410130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:04.411572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:04.412721 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-97bc827c-3f58-401c-bbc8-e2ed7ad8aac5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.163698 Oct 28 11:06:04.413289 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 270/550] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:04 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:05.438125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:05.438733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:05.444565 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:05.445044 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:05.445470 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:05.446180 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:05.446625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:05.447039 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:05.581269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:05.594154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:05.595623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:05.597032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0694820f-fb7e-4cf7-9a1b-4243a3b53c55 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.163346 Oct 28 11:06:05.597594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 281/551] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:05 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:06.622506 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:06.623648 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:06.630211 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:06.630691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:06.631120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:06.631808 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:06.632256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:06.632672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:06.774569 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:06.784580 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:06.786395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:06.787563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7016e38f-2e5c-42fa-9e4d-73c16c936de8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.169003 Oct 28 11:06:06.788297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 271/552] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:06 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:07.813692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:07.814517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:07.820627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:07.821111 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:07.821572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:07.822307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:07.822760 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:07.823179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:07.959497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:07.968338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:07.969788 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:07.970988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-abfba417-e366-4ad4-978f-c04a419afd8c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.161866 Oct 28 11:06:07.971545 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 282/553] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:07 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:08.997036 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:08.998244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:09.004798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:09.005275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:09.005699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:09.006423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:09.006866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:09.007283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:09.132570 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:09.141297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:09.142773 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:09.143926 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-50831100-5461-499d-ac3c-e2a379198cec tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.151007 Oct 28 11:06:09.144484 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 272/554] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:08 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:10.170122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:10.171386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:10.177309 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:10.177817 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:10.178291 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:10.178967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:10.179417 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:10.179839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:10.183661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:06:10.306501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:10.314972 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:10.316449 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:10.317594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-bba238de-b51b-45f8-a458-c10229485e19 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.151547 Oct 28 11:06:10.318186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 283/555] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:10 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:11.343146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:11.343776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:11.350242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:11.350717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:11.351150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:11.351831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:11.352279 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:11.352697 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:11.484500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:11.493161 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:11.494620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:11.495754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-53f03af4-80bb-40d1-99b9-e6858642e66f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.156498 Oct 28 11:06:11.496312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 273/556] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:11 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:12.520951 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:12.521558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:12.527665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:12.528143 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:12.528580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:12.529264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:12.529718 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:12.530173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:12.884602 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:12.893630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:12.895112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:12.896296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f776b642-f259-4158-899c-862c2b58720b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.379242 Oct 28 11:06:12.896851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 284/557] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:12 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:13.921916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:13.922644 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:13.929478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:13.929949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:13.930413 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:13.931098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:13.931552 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:13.931972 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:14.063013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:14.071493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:14.072975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:14.074219 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7acba280-5d2b-4f27-83a8-30278f06a71d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.156226 Oct 28 11:06:14.074784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 274/558] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:13 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:15.100323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:15.100920 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:15.106916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:15.107376 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:15.107803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:15.108986 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:15.109441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:15.109856 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:15.235828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:15.244262 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:15.245726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:15.247033 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0f89fcc8-6e8b-4852-bff5-d4b45f40939f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.150673 Oct 28 11:06:15.247570 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 285/559] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:15 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:16.272093 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:16.273127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:16.279636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:16.280121 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:16.281318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:16.282040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:16.282495 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:16.282946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:16.410294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:16.418730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:16.420152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:16.421311 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6d64e45c-7f50-4278-9e6b-8f1bac0e7173 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.153149 Oct 28 11:06:16.421857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 275/560] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:16 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:17.446332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:17.446948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:17.453446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:17.453911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:17.454370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:17.455041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:17.455484 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:17.455899 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:17.585274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:17.593684 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:17.595130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:17.596265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2f8d8434-00b9-4772-b445-f652096cec95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.153821 Oct 28 11:06:17.596814 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 286/561] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:17 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:17.867837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "24b8315c-e384-4a7f-b0ad-0c6f9ad8e79c"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:06:17.870339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:17.877347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:17.877822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:17.878277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:17.878953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:17.879397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:17.879808 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:18.621872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:18.622525 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:18.628467 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:18.628950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:18.629388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:18.630119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:18.630577 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:18.631002 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:18.755452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:18.764044 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:18.766105 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:18.767266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cdbde8c2-5d8e-42df-81d1-4b24a64c749c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.149290 Oct 28 11:06:18.767823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 287/562] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:18 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:06:19.792735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:19.793395 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:19.800599 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:19.801258 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:19.801734 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:19.802678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:19.803206 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:19.803632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:19.807305 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-709ef9e2-6b04-4da9-9578-2fd1763f69b0 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.943243 Oct 28 11:06:19.807912 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 276/563] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:06:17 2022] POST /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments => generated 194 bytes in 1945 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:06:20.042356 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:20.051525 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:20.052955 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:20.054367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5a1c7bd8-17df-48fd-b674-3b2cd75ce787 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.265386 Oct 28 11:06:20.054959 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 288/564] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:19 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:06:21.082224 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:21.082918 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:21.084838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=30 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:06:21.090286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:21.090764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:21.091197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:21.091881 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:21.092329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:21.093370 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:21.222508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:21.231141 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:21.232590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:21.233900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-974ce6c2-b729-48c5-8606-4216d2d27cb7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.156874 Oct 28 11:06:21.234371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 277/565] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:21 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:22.259778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:22.260320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:22.266698 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:22.267169 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:22.267598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:22.268274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:22.268717 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:22.269130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:22.394313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:22.402781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:22.404205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:22.405906 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d7609979-93a9-4baa-814c-f4b99f5f6b1f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.149541 Oct 28 11:06:22.406482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 289/566] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:22 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:23.293605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-842423e9-d5d0-4135-ab40-dbe1367d0493 req-a1efa8d8-6a3c-405e-8aa3-a0e753122836 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ef9cf5-ad72-4399-a910-9308a243a57a", "name": "network-vif-plugged", "status": "completed", "tag": "06fb27d9-2d31-41b8-ad55-60bad7a8ec79"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:06:23.304398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-842423e9-d5d0-4135-ab40-dbe1367d0493 req-a1efa8d8-6a3c-405e-8aa3-a0e753122836 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:23.304947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-842423e9-d5d0-4135-ab40-dbe1367d0493 req-a1efa8d8-6a3c-405e-8aa3-a0e753122836 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:23.305378 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-842423e9-d5d0-4135-ab40-dbe1367d0493 req-a1efa8d8-6a3c-405e-8aa3-a0e753122836 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:23.316434 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-842423e9-d5d0-4135-ab40-dbe1367d0493 req-a1efa8d8-6a3c-405e-8aa3-a0e753122836 service nova] Creating event network-vif-plugged:06fb27d9-2d31-41b8-ad55-60bad7a8ec79 for instance e4ef9cf5-ad72-4399-a910-9308a243a57a on hv-cinder-zed-3 Oct 28 11:06:23.321143 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-842423e9-d5d0-4135-ab40-dbe1367d0493 req-a1efa8d8-6a3c-405e-8aa3-a0e753122836 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031262 Oct 28 11:06:23.321735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 278/567] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:06:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:06:23.431088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:23.431618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:23.433498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=28 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:06:23.439192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:23.439677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:23.440145 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:23.440837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:23.441296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:23.441747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:23.573364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:23.582776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:23.584212 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:23.585350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9746b8ff-3a70-4dbb-822f-c109253aa8c5 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.158216 Oct 28 11:06:23.585897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 290/568] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:23 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:24.611779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:24.612491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:24.618632 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:24.619111 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:24.619547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:24.620238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:24.620691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:24.621114 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:24.746485 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:24.754926 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:24.756385 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:24.757524 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-54beed38-0206-48ee-84d2-4cda439c355b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.150911 Oct 28 11:06:24.758098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 279/569] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:24 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:25.783197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:25.783731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:25.789635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:25.790149 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:25.790597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:25.791268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:25.791715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:25.792136 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:25.918873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:25.927396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:25.928841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:25.930003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-37cfce1c-c120-4749-9a22-3b9c1872771e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.150750 Oct 28 11:06:25.930568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 291/570] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:25 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:26.955555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:26.956135 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:26.963224 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:26.963726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:26.964139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:26.964837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:26.965280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:26.965756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:27.105739 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:27.114755 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:27.116191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:27.117346 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e3bdebd9-61f8-4b03-9119-51f51707cbcd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1808 microversion: 2.20 time: 0.166379 Oct 28 11:06:27.117903 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 280/571] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:26 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:27.341678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:06:27.344099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:27.350067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:27.350554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:27.350984 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:27.351664 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:27.352116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:27.352537 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:27.374896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lazy-loading 'flavor' on Instance uuid 01a19441-142f-41aa-9253-c5b50c929e93 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:06:27.491719 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-76865181-3725-417c-a226-25ef99b0697a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/action" status: 200 len: 32 microversion: 2.1 time: 0.153787 Oct 28 11:06:27.492161 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 292/572] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:06:27 2022] POST /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/action => generated 32 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:06:27.502763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:27.503376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:27.509355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:27.509912 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:27.510442 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:27.511164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:27.511688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:27.512858 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:27.651634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:27.665817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5eb63273-10d4-4965-9270-4488d0d5a9b9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.166621 Oct 28 11:06:27.666965 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 281/573] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:27 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:28.246809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:28.247527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:28.254871 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:28.255344 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:28.255776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:28.256457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:28.256902 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:28.257314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:28.385396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:28.394751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:28.396199 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:28.397355 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9dd07fb9-6952-4cc1-b6b1-28e918ca1ea9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1795 microversion: 2.20 time: 0.255848 Oct 28 11:06:28.397917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 293/574] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:06:28 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1795 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:06:28.793222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:28.793830 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:28.800000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:28.800475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:28.800903 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:28.801585 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:28.802082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:28.802508 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:28.930250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:28.938896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ec79e779-f90f-4bbf-b2f5-1ceb63d06ebd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.254238 Oct 28 11:06:28.939465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 282/575] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:28 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:29.962506 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:29.963108 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:29.969228 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:29.969709 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:29.970167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:29.970870 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:29.971319 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:29.971756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:30.100057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:30.108191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-80afd913-d537-4bdb-9869-e15b22096d89 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.149558 Oct 28 11:06:30.109293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 294/576] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:29 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:31.132848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:31.133392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:31.139754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:31.140225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:31.140651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:31.141323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:31.141767 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:31.142209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:31.144768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:06:31.270255 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:31.278393 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a380ebca-a876-4dde-b27e-89b66695e144 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.149424 Oct 28 11:06:31.278954 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 283/577] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:31 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:32.302413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:32.303021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:32.308889 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:32.309364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:32.309796 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:32.310502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:32.310951 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:32.311367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:32.435613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:32.444287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d8608744-d164-4972-b987-21288a3af5d9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.145912 Oct 28 11:06:32.444852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 295/578] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:32 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:33.468265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:33.468861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:33.471813 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 28 11:06:33.471959 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Fri Oct 28 11:06:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 (ip 10.222.0.46) !!! Oct 28 11:06:33.483990 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:33.484461 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:33.484882 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:33.485824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:33.486295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:33.486709 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:33.616945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:33.625187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-11f1bd24-afde-4cd3-8de6-5db8ceca5f65 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.161081 Oct 28 11:06:33.625751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 284/579] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:33 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:34.648676 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:34.649284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:34.656048 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:34.656524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:34.656955 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:34.657638 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:34.658121 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:34.658547 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:34.781856 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:34.789933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2a7be664-f69d-4a45-89be-af04fb62a4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.145139 Oct 28 11:06:34.790519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 296/580] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:34 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:06:35.813139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:35.813747 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:35.820150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:35.820623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:35.821054 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:35.821735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:35.822224 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:35.822645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:35.947277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:35.955955 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e990d536-4d59-4134-8c07-dadbc5d75d4a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.146765 Oct 28 11:06:35.956515 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 285/581] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:35 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:36.979603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:36.980207 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:36.987026 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:36.987514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:36.987919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:36.988591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:36.989037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:36.989451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:37.116141 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:37.118478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:06:37.126730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-16e6b143-4421-477c-ba80-c1722cd6a1f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.150990 Oct 28 11:06:37.127301 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 297/582] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:36 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:38.150173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:38.150780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:38.157772 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:38.158274 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:38.158708 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:38.159391 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:38.159842 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:38.160263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:38.288150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:38.296234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ed2620fc-6828-4765-89e3-ddd1852f706a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.149980 Oct 28 11:06:38.296801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 286/583] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:38 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:39.320406 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:39.321054 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:39.329506 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:39.330074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:39.330510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:39.331194 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:39.331643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:39.332064 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:39.458928 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:39.467125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-358d18bf-4325-4e52-91a3-fa6ed9318d6b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.150568 Oct 28 11:06:39.467873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 298/584] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:39 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:40.490440 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:40.491090 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:40.497547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:40.498051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:40.498487 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:40.499777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:40.500224 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:40.500646 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:40.628621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:40.636651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ff866a4e-8b08-4041-82d9-5e1460348928 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.150081 Oct 28 11:06:40.637205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 287/585] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:40 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:41.660302 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:41.660918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:41.666913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:41.667396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:41.667829 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:41.668513 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:41.668970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:41.669395 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:41.796634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:41.804834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7506e3c9-9e67-4c29-a94f-9505e4ac0c58 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.148510 Oct 28 11:06:41.805398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 299/586] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:41 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:42.828400 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:42.829008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:42.835286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:42.835752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:42.836179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:42.836853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:42.837300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:42.837712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:42.963136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:42.971990 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-eae313b2-43bc-4358-9dec-6f0a739a22cc tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.147913 Oct 28 11:06:42.972551 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 288/587] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:42 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:43.996030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:43.996591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:44.002372 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:44.002842 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:44.003269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:44.003941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:44.004395 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:44.004805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:44.129603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:44.137771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-53198b4d-b270-445e-b886-eb71d6d28bc6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.145736 Oct 28 11:06:44.138384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 300/588] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:43 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:45.160708 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:45.161318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:45.168192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:45.168669 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:45.169097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:45.169778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:45.170256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:45.170677 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:45.299635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:45.307714 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f60a2780-0dc2-483f-899d-0a0cc6b18f17 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.150922 Oct 28 11:06:45.308270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 289/589] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:45 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:46.331790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:46.332407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:46.338530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:46.339007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:46.339440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:46.340124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:46.340576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:46.341000 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:46.465356 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:46.474155 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4fa73d93-25fd-48a8-ae58-fe6616f89f5b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1735 microversion: 2.1 time: 0.146701 Oct 28 11:06:46.474725 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 301/590] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:46 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1735 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:47.497459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:47.498091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:47.504871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.505341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.505770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:47.506472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.506923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.507339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:47.632671 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:47.641314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e51938f4-02c5-4627-ab92-b44b708424f6 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1715 microversion: 2.1 time: 0.147699 Oct 28 11:06:47.641869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 290/591] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:47 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1715 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:47.701341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:47.701839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:47.708429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.708909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.709346 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:47.710063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.710525 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.710946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:47.832740 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] HTTP exception thrown: Cannot 'detach_volume' instance 01a19441-142f-41aa-9253-c5b50c929e93 while it is in vm_state rescued Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Request method failure captured: Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments/24b8315c-e384-4a7f-b0ad-0c6f9ad8e79c HTTP/1.1 Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7aIhPtnefCCqNmAPadWpk5584o2_2r0isTmVkRGKzWI-I7c1QbiFIdoruXOy5h0VZogX9L4sPi-DVMWkisxvUM6avn2ESWp6fMy2fmzi6DazAAqVi14U8Y2drfllWP-AiHWNP38WDQIVdY8aOS1D9QC35crpyrYbxvM8s51pB0LFgc Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:06:47.834336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 347e262907da4a83a197aaf42cecf51d Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:06:47.835942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Cannot 'detach_volume' instance 01a19441-142f-41aa-9253-c5b50c929e93 while it is in vm_state rescued Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'server_id': '01a19441-142f-41aa-9253-c5b50c929e93', 'id': '24b8315c-e384-4a7f-b0ad-0c6f9ad8e79c'} Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 01a19441-142f-41aa-9253-c5b50c929e93 in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:06:47.836747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 01a19441-142f-41aa-9253-c5b50c929e93 while it is in vm_state rescued Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 01a19441-142f-41aa-9253-c5b50c929e93 while it is in vm_state rescued Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:06:47.838345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Returning 409 to user: Cannot 'detach_volume' instance 01a19441-142f-41aa-9253-c5b50c929e93 while it is in vm_state rescued {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:06:47.839572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8474f39c-f83b-41a1-b246-191d82ed4432 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "DELETE /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments/24b8315c-e384-4a7f-b0ad-0c6f9ad8e79c" status: 409 len: 152 microversion: 2.1 time: 0.138505 Oct 28 11:06:47.839572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 302/592] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:06:47 2022] DELETE /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments/24b8315c-e384-4a7f-b0ad-0c6f9ad8e79c => generated 152 bytes in 140 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 28 11:06:47.844726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:06:47.845246 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:47.851022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.851494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.851922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:47.852599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.853042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.853457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:47.915606 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0dce860d-4230-4f53-9f80-77b719953971 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "POST /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/action" status: 202 len: 0 microversion: 2.1 time: 0.074556 Oct 28 11:06:47.916242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 291/593] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:06:47 2022] POST /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/action => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:06:47.925325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:47.926037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:47.931906 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.932373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.932803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:47.933487 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:47.933933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:47.934375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:48.144559 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:48.152642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9c4a6117-7970-42b9-b879-77249342c4ae tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1723 microversion: 2.1 time: 0.232177 Oct 28 11:06:48.153429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 303/594] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:47 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1723 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:49.177915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:49.178636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:49.184696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:49.185174 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:49.185603 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:49.186317 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:49.186769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:49.187194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:49.312866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:49.320914 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-586d5508-3f80-40d5-85c3-c9e1e2edc4a5 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1723 microversion: 2.1 time: 0.147726 Oct 28 11:06:49.321471 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 292/595] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:49 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1723 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:50.344929 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:50.345529 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:50.351414 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:50.351887 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:50.352340 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:50.353021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:50.353466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:50.353880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:50.480930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:50.489950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b028b124-7047-473d-8134-a0fccbc8f984 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1723 microversion: 2.1 time: 0.149462 Oct 28 11:06:50.490632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 304/596] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:50 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:51.512887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:51.513490 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:51.516067 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=32 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:06:51.521234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:51.521704 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:51.522169 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:51.522895 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:51.523289 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:51.523705 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:51.661099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:51.669240 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2a79da1e-60a6-4b1e-a7b4-c7bb6238fd30 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1729 microversion: 2.1 time: 0.160502 Oct 28 11:06:51.669811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 293/597] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:51 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1729 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:51.691234 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:51.691775 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:51.697448 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:51.697917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:51.698374 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:51.699045 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:51.699487 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:51.699907 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:51.774512 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-02837f50-90a6-4242-8620-dd4d87c20055 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.086904 Oct 28 11:06:51.775046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 305/598] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:06:51 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-interface => generated 284 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:06:52.673768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-afe1654b-ab1c-431b-8616-c90101068084 req-d264c935-457d-4fb5-a824-b4465edb7469 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01a19441-142f-41aa-9253-c5b50c929e93", "tag": "eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:06:52.682985 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-afe1654b-ab1c-431b-8616-c90101068084 req-d264c935-457d-4fb5-a824-b4465edb7469 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:52.683452 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-afe1654b-ab1c-431b-8616-c90101068084 req-d264c935-457d-4fb5-a824-b4465edb7469 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:52.683869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-afe1654b-ab1c-431b-8616-c90101068084 req-d264c935-457d-4fb5-a824-b4465edb7469 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:52.694815 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-afe1654b-ab1c-431b-8616-c90101068084 req-d264c935-457d-4fb5-a824-b4465edb7469 service nova] Creating event network-changed:eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f for instance 01a19441-142f-41aa-9253-c5b50c929e93 on hv-cinder-zed-3 Oct 28 11:06:52.699904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-afe1654b-ab1c-431b-8616-c90101068084 req-d264c935-457d-4fb5-a824-b4465edb7469 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087076 Oct 28 11:06:52.700602 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 294/599] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:06:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:06:54.309677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:06:54.312492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:54.314371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=32 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:06:54.319765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:54.320242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:54.320678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:54.321358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:54.321813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:54.322279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:54.391354 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-42f1244f-44a0-4b9b-947a-39b1216f0b43 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/action" status: 202 len: 0 microversion: 2.1 time: 0.085425 Oct 28 11:06:54.391963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 306/600] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:06:54 2022] POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/action => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:06:54.400927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:54.401696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:54.407824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:54.408308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:54.408726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:54.409404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:54.409847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:54.410287 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:54.553241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:54.562342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7b264f56-82a7-4e59-b256-294db67bf265 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1751 microversion: 2.1 time: 0.165372 Oct 28 11:06:54.563036 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 295/601] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:54 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1751 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:54.716507 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-bec0fcc1-edfc-40af-bca5-64b3d85dd1aa req-cb767dc0-4b48-431b-b566-7cfc581416e2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01a19441-142f-41aa-9253-c5b50c929e93", "tag": "eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:06:54.725998 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-bec0fcc1-edfc-40af-bca5-64b3d85dd1aa req-cb767dc0-4b48-431b-b566-7cfc581416e2 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:54.726468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-bec0fcc1-edfc-40af-bca5-64b3d85dd1aa req-cb767dc0-4b48-431b-b566-7cfc581416e2 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:54.726889 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-bec0fcc1-edfc-40af-bca5-64b3d85dd1aa req-cb767dc0-4b48-431b-b566-7cfc581416e2 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:54.738616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-bec0fcc1-edfc-40af-bca5-64b3d85dd1aa req-cb767dc0-4b48-431b-b566-7cfc581416e2 service nova] Creating event network-changed:eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f for instance 01a19441-142f-41aa-9253-c5b50c929e93 on hv-cinder-zed-3 Oct 28 11:06:54.743592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-bec0fcc1-edfc-40af-bca5-64b3d85dd1aa req-cb767dc0-4b48-431b-b566-7cfc581416e2 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031107 Oct 28 11:06:54.744195 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 307/602] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:06:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:06:55.586334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:55.586938 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:55.593917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:55.594427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:55.594852 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:55.595804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:55.596256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:55.596672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:55.737259 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:55.745343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3be24d13-ef79-4b75-ba7b-8d34cece0011 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.162882 Oct 28 11:06:55.745897 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 296/603] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:55 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:56.770122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:56.770840 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:56.776831 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:56.777298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:56.777723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:56.778419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:56.778864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:56.779275 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:56.920739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:56.928800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-37935706-7b7d-4b26-8327-6b4f0775b25c tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.163026 Oct 28 11:06:56.929356 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 308/604] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:56 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:57.951000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:57.951537 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:57.957891 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:57.958394 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:57.958822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:57.959498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:57.959976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:57.960391 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:58.098409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:58.106595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-648ca592-60ec-474a-833b-b533391b25f4 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.159433 Oct 28 11:06:58.107157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 297/605] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:57 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:06:59.131044 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:06:59.131678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:06:59.137856 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:59.138360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:59.138787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:59.139485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:06:59.140436 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:06:59.140849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:06:59.278691 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:06:59.287639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b01116b2-ad4b-43c2-95d5-4f83a3cc0b32 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.160522 Oct 28 11:06:59.288235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 309/606] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:06:59 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:00.310079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:00.310628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:00.317018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:00.317491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:00.317919 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:00.318626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:00.319108 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:00.319525 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:00.458380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:00.466500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-45bd6438-6d11-439c-8dac-2b3b9762827b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.160336 Oct 28 11:07:00.467058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 298/607] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:00 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:01.490053 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:01.490591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:01.496408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:01.496875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:01.497300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:01.497985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:01.498437 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:01.498854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:01.641393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:01.649489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-506b95f9-4923-4daf-b385-e6b1eb802f6b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.163634 Oct 28 11:07:01.650070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 310/608] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:01 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:02.672425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:02.673032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:02.679039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:02.679506 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:02.679936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:02.680613 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:02.681060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:02.681473 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:02.684052 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:02.821088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:02.829137 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-90b9e51e-981c-4104-905b-382d63f0d203 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.160602 Oct 28 11:07:02.829697 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 299/609] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:02 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:03.852725 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:03.853326 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:03.859287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:03.859757 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:03.860186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:03.861133 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:03.861578 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:03.862015 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:03.910393 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:03.910952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:03.917571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:03.918080 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:03.918512 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:03.919203 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:03.919654 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:03.920077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:03.945262 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:07:04.003218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:04.011480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-21c912f7-e333-46f9-b981-f5193b3c8b3e tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.162668 Oct 28 11:07:04.012047 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 311/610] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:03 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:04.027043 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c162b36f-5c8e-4c5d-9167-edfe9ac53ae7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "DELETE /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 204 len: 0 microversion: 2.1 time: 0.120003 Oct 28 11:07:04.027757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 300/611] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:07:03 2022] DELETE /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 11:07:04.036018 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:04.036703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:04.043043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:04.043515 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:04.043944 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:04.044625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:04.045075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:04.045921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:04.173839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:04.182056 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-96480e76-28cf-4118-8c90-4d19ff154331 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1578 microversion: 2.1 time: 0.149538 Oct 28 11:07:04.182623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 312/612] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:04 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1578 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:07:05.036270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:05.036971 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:05.043731 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:05.044220 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:05.044652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:05.045341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:05.045803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:05.046249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:05.190355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:05.198694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-56a629d8-895e-4834-8f74-45cac7c4aac2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.166689 Oct 28 11:07:05.199411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 301/613] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:05 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:05.204588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:05.205311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:05.211297 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:05.211841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:05.212361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:05.213106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:05.213629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:05.214151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:05.342812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:05.350901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3bf58b6f-0c34-4513-b3a4-8f35ba989d80 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1578 microversion: 2.1 time: 0.150196 Oct 28 11:07:05.351459 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 313/614] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:05 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1578 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:06.221668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:06.222301 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:06.228656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:06.229132 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:06.229852 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:06.230555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:06.231000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:06.231417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:06.374661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:06.375961 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:06.381812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:06.382326 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:06.382761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:06.383447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:06.383901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:06.384324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:06.399716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:06.464847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:06.474433 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e452b56d-9a01-410e-8351-fd2665f36e4a tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.256454 Oct 28 11:07:06.475777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 302/615] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:06 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:06.520284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:06.528200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c74cd048-b078-48c3-b9d7-b428c11fddcc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1578 microversion: 2.1 time: 0.157312 Oct 28 11:07:06.528771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 314/616] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:06 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1578 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:07:07.152498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-a07b3466-c52b-4be4-92a1-db62705e0423 req-1a67ad42-2af2-4363-a542-6b463c42de0b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "917e11e6-71de-4156-bffc-7a5a81135795", "name": "network-vif-deleted", "tag": "4fc47c92-3eb6-4a2b-85be-a8582c825e05"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:07.165896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a07b3466-c52b-4be4-92a1-db62705e0423 req-1a67ad42-2af2-4363-a542-6b463c42de0b service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:07.166486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a07b3466-c52b-4be4-92a1-db62705e0423 req-1a67ad42-2af2-4363-a542-6b463c42de0b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:07.167118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a07b3466-c52b-4be4-92a1-db62705e0423 req-1a67ad42-2af2-4363-a542-6b463c42de0b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:07.179902 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-a07b3466-c52b-4be4-92a1-db62705e0423 req-1a67ad42-2af2-4363-a542-6b463c42de0b service nova] Creating event network-vif-deleted:4fc47c92-3eb6-4a2b-85be-a8582c825e05 for instance 917e11e6-71de-4156-bffc-7a5a81135795 on hv-cinder-zed-3 Oct 28 11:07:07.185275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-a07b3466-c52b-4be4-92a1-db62705e0423 req-1a67ad42-2af2-4363-a542-6b463c42de0b service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099821 Oct 28 11:07:07.186062 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 303/617] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:07.499513 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:07.501168 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:07.507310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:07.507863 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:07.508384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:07.509351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:07.510067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:07.510696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:07.552715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:07.553408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:07.560390 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:07.561154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:07.561821 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:07.563418 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:07.564105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:07.564745 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:07.598396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:07.599197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:07.600355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:07.601121 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lazy-loading 'fault' on Instance uuid 917e11e6-71de-4156-bffc-7a5a81135795 {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:07:07.646539 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:07.663747 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-35999fda-b8aa-41fe-8a6e-afce11f413de tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 200 len: 1231 microversion: 2.1 time: 0.117098 Oct 28 11:07:07.664805 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 304/618] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:07 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 1231 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:07.678042 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:07.686764 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e219efbc-58fe-4ed8-91fe-6b3bca413f9d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.192255 Oct 28 11:07:07.687385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 315/619] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:07 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:08.688443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:08.689186 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:08.695956 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:08.696444 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:08.696878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:08.697587 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:08.698104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:08.698529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:08.710220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:08.710904 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:08.714621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] HTTP exception thrown: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Request method failure captured: Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 HTTP/1.1 Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7bpKhsLly_AsLC5I5Ufi7RRtWwekiZ20oTVh9O4umzrCT6TaV8PH5o_R9SYBAOVP2EofrgrO2OiTcdq11VWJ9GiBna5aI-gXnS1_CYhIzgxrlZivt_OgETCTDDtqo0G4RtPvy7n959ltOqSgWxONV0J0lU0ZrvO6vH8e0oBtYQ2BtI Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:07:08.716435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 6b926f7161c0487a8c2b04ada3b374e3 Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:07:08.717825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '917e11e6-71de-4156-bffc-7a5a81135795'} Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:07:08.718710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:07:08.719958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:08.721088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Returning 404 to user: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:07:08.721176 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:08.721263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d0f21353-277d-4752-bf50-0a83656c43dd tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 404 len: 111 microversion: 2.1 time: 0.034086 Oct 28 11:07:08.721337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:08.721423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 305/620] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:08 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:07:08.721498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:08.721498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:08.721498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:08.721498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:08.825555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1302927279", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1118103513", "security_groups": [{"name": "tempest-secgroup-smoke-1525806302"}], "block_device_mapping_v2": [{"uuid": "0fadb5f1-7cd6-43db-a7e6-19f02c8df548", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "447ffcd3-d772-4c15-92cb-558dd17897d4"}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:08.879308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:08.887640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1fc2f9f0-a783-4b4d-a619-6df891ff7338 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.181399 Oct 28 11:07:08.888209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 316/621] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:08 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:07:08.951670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] validate_networks() for [('447ffcd3-d772-4c15-92cb-558dd17897d4', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:07:09.356848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:07:09.357272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:07:09.357742 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:07:09.358455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:07:09.358871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:07:09.368515 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:09.369005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:09.369443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:09.370278 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:09.370696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:09.371124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:09.388020 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:07:09.392496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for user 6b926f7161c0487a8c2b04ada3b374e3 and project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:07:09.450629 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:07:09.453578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='0fadb5f1-7cd6-43db-a7e6-19f02c8df548',volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:07:09.594200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0650e071-32ac-44ab-a7bd-2f7d7fa172da tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.771805 Oct 28 11:07:09.594839 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 306/622] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:07:08 2022] POST /compute/v2.1/servers => generated 406 bytes in 774 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 28 11:07:09.605753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b8c8ef4d-6479-4942-9e78-711774ff1a5e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:09.606391 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b8c8ef4d-6479-4942-9e78-711774ff1a5e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:09.661197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b8c8ef4d-6479-4942-9e78-711774ff1a5e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:09.668287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b8c8ef4d-6479-4942-9e78-711774ff1a5e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1095 microversion: 2.1 time: 0.066263 Oct 28 11:07:09.668893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 317/623] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:09 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1095 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:09.911832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:09.913024 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:09.920419 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:09.920905 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:09.921333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:09.922907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:09.923359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:09.923779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:10.075746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:10.084907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ace662ac-c933-4faf-955b-1bee656c5f1f tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1756 microversion: 2.1 time: 0.177341 Oct 28 11:07:10.085455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 307/624] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:09 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1756 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:07:10.691499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:10.692043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:10.698114 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:10.698607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:10.699038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:10.699723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:10.700208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:10.700630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:10.764896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:10.773055 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6172696b-47d2-437d-bf1a-f2ae122ed73c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1193 microversion: 2.1 time: 0.085995 Oct 28 11:07:10.773612 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 318/625] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:10 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1193 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:11.108615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:11.109215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:11.115896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:11.116389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:11.116812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:11.117484 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:11.117934 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:11.118372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:11.255027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:11.263056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-cfe06910-e226-4c65-a307-8686dbd1836d tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1759 microversion: 2.1 time: 0.158308 Oct 28 11:07:11.263609 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 308/626] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:11 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1759 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:11.797077 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:11.797827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:11.804706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:11.805491 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:11.805892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:11.806641 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:11.807104 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:11.807538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:11.872882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:11.881343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-76a969e4-3d31-4086-b1da-254fa6d6dd17 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1211 microversion: 2.1 time: 0.088432 Oct 28 11:07:11.881912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 319/627] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:11 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1211 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:12.749505 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-5bc7b434-050b-48ad-8ec6-6132a0112e86 req-f08d9779-f988-4def-9183-4f45bcf51e7b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a", "tag": "77a42644-3142-4fb2-9d0e-5deb0792310a"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:12.761222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-5bc7b434-050b-48ad-8ec6-6132a0112e86 req-f08d9779-f988-4def-9183-4f45bcf51e7b service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:12.761685 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-5bc7b434-050b-48ad-8ec6-6132a0112e86 req-f08d9779-f988-4def-9183-4f45bcf51e7b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:12.762357 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-5bc7b434-050b-48ad-8ec6-6132a0112e86 req-f08d9779-f988-4def-9183-4f45bcf51e7b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:12.773481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-5bc7b434-050b-48ad-8ec6-6132a0112e86 req-f08d9779-f988-4def-9183-4f45bcf51e7b service nova] Creating event network-changed:77a42644-3142-4fb2-9d0e-5deb0792310a for instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a on hv-cinder-zed-3 Oct 28 11:07:12.778797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-5bc7b434-050b-48ad-8ec6-6132a0112e86 req-f08d9779-f988-4def-9183-4f45bcf51e7b service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033010 Oct 28 11:07:12.779577 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 309/628] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:12.904611 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:12.905226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:12.911716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:12.912748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:12.913196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:12.913893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:12.914384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:12.914813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:13.042921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:13.051096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-326de447-b27c-4125-88db-66f5c29c1e61 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.150335 Oct 28 11:07:13.051670 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 320/629] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:12 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:14.075596 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:14.076891 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:14.083780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:14.084272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:14.084713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:14.085414 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:14.085900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:14.086349 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:14.298339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:14.306225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a00b731b-5976-4d6c-8e9b-ebeed321c01a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.235749 Oct 28 11:07:14.306792 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 310/630] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:14 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:15.329014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:15.329613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:15.335581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:15.336053 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:15.336478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:15.337149 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:15.337595 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:15.338028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:15.467514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:15.477408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8c04f97e-78a6-4726-b068-ed0fa3d254ea tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.152237 Oct 28 11:07:15.477992 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 321/631] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:15 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:16.500123 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:16.500730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:16.507894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:16.508371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:16.508805 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:16.509494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:16.509944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:16.510388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:16.643902 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:16.651546 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2a1d7220-db09-4a2c-9b92-4e1191babbe5 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.155293 Oct 28 11:07:16.652095 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 311/632] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:16 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:17.673533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:17.674171 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:17.680250 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:17.680726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:17.681158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:17.681836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:17.682326 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:17.682744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:17.809511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:17.817090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-181da232-29f6-4954-aab9-c68713cd0d03 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.148047 Oct 28 11:07:17.817649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 322/633] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:17 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:18.840165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:18.840770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:18.847752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:18.848223 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:18.848648 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:18.849318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:18.849762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:18.850204 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:18.975768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:18.983260 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-495f81e4-397a-4161-8b8f-87a83c40b061 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.147013 Oct 28 11:07:18.983829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 312/634] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:18 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:20.007759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:20.008379 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:20.010405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:20.015635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:20.016108 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:20.016540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:20.017224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:20.017675 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:20.018132 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:20.143794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:20.151386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4f7ff729-f6a6-4d34-9276-c648aafbd902 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.147446 Oct 28 11:07:20.151945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 323/635] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:20 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:21.173735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:21.174346 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:21.176358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:21.177171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:21.183883 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:21.184358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:21.184790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:21.185477 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:21.185927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:21.186369 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:21.316231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:21.323786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2f7e880b-c2c1-44d4-b95c-dc0359fc8bd8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.153995 Oct 28 11:07:21.324349 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 313/636] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:21 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:22.346854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:22.347465 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:22.353722 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:22.354248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:22.354684 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:22.355365 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:22.355815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:22.356235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:22.483729 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:22.491516 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d17991ee-9230-45c4-bca7-3f67e8327a35 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.149144 Oct 28 11:07:22.492087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 324/637] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:22 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:23.514738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:23.515347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:23.518308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=62,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:23.523906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:23.524316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:23.524771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:23.525450 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:23.525895 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:23.526341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:23.659649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:23.668020 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0ddd85c4-90d4-41c2-abd4-61ff0f534846 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.157362 Oct 28 11:07:23.668625 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 314/638] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:23 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:23.690488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-c504c863-09e4-4585-8504-0bdd7cfb281e req-6e5fd71b-d1f2-46ab-8457-bc652d6882bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a", "name": "network-vif-plugged", "status": "completed", "tag": "77a42644-3142-4fb2-9d0e-5deb0792310a"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:23.699142 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c504c863-09e4-4585-8504-0bdd7cfb281e req-6e5fd71b-d1f2-46ab-8457-bc652d6882bf service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:23.699615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c504c863-09e4-4585-8504-0bdd7cfb281e req-6e5fd71b-d1f2-46ab-8457-bc652d6882bf service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:23.700035 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c504c863-09e4-4585-8504-0bdd7cfb281e req-6e5fd71b-d1f2-46ab-8457-bc652d6882bf service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:23.710749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-c504c863-09e4-4585-8504-0bdd7cfb281e req-6e5fd71b-d1f2-46ab-8457-bc652d6882bf service nova] Creating event network-vif-plugged:77a42644-3142-4fb2-9d0e-5deb0792310a for instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a on hv-cinder-zed-3 Oct 28 11:07:23.715522 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-c504c863-09e4-4585-8504-0bdd7cfb281e req-6e5fd71b-d1f2-46ab-8457-bc652d6882bf service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102724 Oct 28 11:07:23.716097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 325/639] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:23.991012 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:23.991533 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:23.997923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:23.998434 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:23.998864 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:23.999549 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:23.999998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:24.001014 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:24.031681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3d2662a5-8891-4cd8-bacb-583002f622a0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments/47acd41a-f217-45c0-88d0-aca5ec5373f9" status: 200 len: 194 microversion: 2.20 time: 0.044605 Oct 28 11:07:24.032277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 315/640] 10.222.0.46 () {60 vars in 1454 bytes} [Fri Oct 28 11:07:23 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments/47acd41a-f217-45c0-88d0-aca5ec5373f9 => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 11:07:24.100476 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:24.101042 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:24.107071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:24.107545 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:24.108027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:24.108706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:24.109147 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:24.110190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:24.188852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lazy-loading 'services' on Instance uuid e4ef9cf5-ad72-4399-a910-9308a243a57a {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:07:24.298610 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3a9fd9ba-be1f-4cd7-afb4-c5619f37c4d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments/47acd41a-f217-45c0-88d0-aca5ec5373f9" status: 202 len: 0 microversion: 2.20 time: 0.202185 Oct 28 11:07:24.299186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 326/641] 10.222.0.46 () {60 vars in 1457 bytes} [Fri Oct 28 11:07:24 2022] DELETE /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments/47acd41a-f217-45c0-88d0-aca5ec5373f9 => generated 0 bytes in 204 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:07:24.307471 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:24.309616 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:24.315488 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:24.315967 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:24.316400 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:24.317103 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:24.317549 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:24.317968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:24.331555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:24.350570 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-61fd8153-11ee-4e5d-8160-7d07603f8304 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046976 Oct 28 11:07:24.351126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 316/642] 10.222.0.46 () {60 vars in 1343 bytes} [Fri Oct 28 11:07:24 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 11:07:24.691742 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:24.692428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:24.694359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=44 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:24.699706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:24.700177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:24.700606 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:24.701278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:24.701722 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:24.702177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:24.829470 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:24.837249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d065360e-48fc-4c33-ac7a-9825d8adf489 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.149641 Oct 28 11:07:24.838040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 327/643] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:24 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:25.363446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:25.365547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:25.371944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:25.372421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:25.372846 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:25.373517 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:25.373957 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:25.374395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:25.403973 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-547386da-e886-491b-8e76-eb1558e29e14 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044704 Oct 28 11:07:25.404607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 317/644] 10.222.0.46 () {60 vars in 1343 bytes} [Fri Oct 28 11:07:25 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 11:07:25.859981 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:25.860631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:25.866774 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:25.867245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:25.867669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:25.868347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:25.868794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:25.869210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:25.999942 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:26.008364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-09fd3b32-e1ed-485b-a4a4-02a312f8d83e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1267 microversion: 2.1 time: 0.152331 Oct 28 11:07:26.008931 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 328/645] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:25 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:26.417232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:26.419377 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:26.426288 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:26.426756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:26.427181 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:26.427853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:26.428296 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:26.428710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:26.456134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7a7eac8c-4638-43ac-940f-569270781bd9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.042892 Oct 28 11:07:26.456675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 318/646] 10.222.0.46 () {60 vars in 1343 bytes} [Fri Oct 28 11:07:26 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:26.519434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:26.519927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:26.525750 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:26.526217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:26.526648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:26.527321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:26.527774 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:26.528190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:26.553089 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:07:26.627503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5740e068-11ec-4578-89c1-cefab33cacb8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 204 len: 0 microversion: 2.20 time: 0.111573 Oct 28 11:07:26.628130 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 329/647] 10.222.0.46 () {60 vars in 1280 bytes} [Fri Oct 28 11:07:26 2022] DELETE /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 28 11:07:26.636292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:26.636900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:26.642786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:26.643257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:26.643687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:26.644367 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:26.644814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:26.645241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:26.771040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:26.779139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:26.780527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:26.782328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3dcb27b2-1165-4b28-9381-fb46df8719ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.149274 Oct 28 11:07:26.782888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 319/648] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:26 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:07:27.032151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:27.032781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:27.038661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:27.039138 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:27.039568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:27.040245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:27.040698 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:27.041116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:27.055462 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:27.171704 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:27.179277 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-13204a6b-5775-44ca-9ff9-4fac99ec8a06 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1454 microversion: 2.1 time: 0.151992 Oct 28 11:07:27.179843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 330/649] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:27 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1454 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:27.201423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:27.202075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:27.208560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:27.209035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:27.209466 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:27.210175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:27.210632 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:27.211039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:27.337896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:27.345730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-865f843a-d8c4-4721-ba98-3246757c8e23 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1454 microversion: 2.1 time: 0.148201 Oct 28 11:07:27.346331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 320/650] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:27 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1454 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:27.807576 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:27.808225 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:27.814710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:27.815197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:27.815632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:27.816318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:27.816771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:27.817196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:27.946773 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:27.966129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:27.968714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:27.969930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8c772442-254e-41fa-97f8-48c084227844 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1723 microversion: 2.20 time: 0.166294 Oct 28 11:07:27.970565 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 331/651] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:27 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:28.996872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:28.998356 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:29.004802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:29.005359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:29.005868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:29.006683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:29.007215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:29.007836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:29.109070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:29.118527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:29.124762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:29.126078 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-98424dea-5466-4774-891f-37af42f3f35c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1656 microversion: 2.20 time: 0.133227 Oct 28 11:07:29.126641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 321/652] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:28 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1656 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:29.396509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-489386d6-e04c-4daa-a51e-1668dab0570e req-cc53995c-0782-4482-9795-a20e722a7b61 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ef9cf5-ad72-4399-a910-9308a243a57a", "name": "network-vif-deleted", "tag": "06fb27d9-2d31-41b8-ad55-60bad7a8ec79"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:29.417948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-489386d6-e04c-4daa-a51e-1668dab0570e req-cc53995c-0782-4482-9795-a20e722a7b61 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:29.420561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-489386d6-e04c-4daa-a51e-1668dab0570e req-cc53995c-0782-4482-9795-a20e722a7b61 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:29.421065 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-489386d6-e04c-4daa-a51e-1668dab0570e req-cc53995c-0782-4482-9795-a20e722a7b61 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:29.439361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-489386d6-e04c-4daa-a51e-1668dab0570e req-cc53995c-0782-4482-9795-a20e722a7b61 service nova] Creating event network-vif-deleted:06fb27d9-2d31-41b8-ad55-60bad7a8ec79 for instance e4ef9cf5-ad72-4399-a910-9308a243a57a on hv-cinder-zed-3 Oct 28 11:07:29.446884 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-489386d6-e04c-4daa-a51e-1668dab0570e req-cc53995c-0782-4482-9795-a20e722a7b61 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053911 Oct 28 11:07:29.447636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 332/653] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:29.697594 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-ace802f0-914d-4920-828b-014c0a38c663 req-328ad5ea-e691-4e0e-94b3-95d7d3b19c6b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a", "tag": "77a42644-3142-4fb2-9d0e-5deb0792310a"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:29.712487 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ace802f0-914d-4920-828b-014c0a38c663 req-328ad5ea-e691-4e0e-94b3-95d7d3b19c6b service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:29.713187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ace802f0-914d-4920-828b-014c0a38c663 req-328ad5ea-e691-4e0e-94b3-95d7d3b19c6b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:29.713825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ace802f0-914d-4920-828b-014c0a38c663 req-328ad5ea-e691-4e0e-94b3-95d7d3b19c6b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:29.726501 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-ace802f0-914d-4920-828b-014c0a38c663 req-328ad5ea-e691-4e0e-94b3-95d7d3b19c6b service nova] Creating event network-changed:77a42644-3142-4fb2-9d0e-5deb0792310a for instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a on hv-cinder-zed-3 Oct 28 11:07:29.733119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-ace802f0-914d-4920-828b-014c0a38c663 req-328ad5ea-e691-4e0e-94b3-95d7d3b19c6b service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039477 Oct 28 11:07:29.734389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 322/654] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:30.154007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:30.154715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:30.170375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:30.171043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:30.172174 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:30.177885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:30.178828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:30.179529 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:30.229031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:30.229769 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:30.230383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:30.230985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lazy-loading 'fault' on Instance uuid e4ef9cf5-ad72-4399-a910-9308a243a57a {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:07:30.300698 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:30.309036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:30.310554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:30.312000 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1a799c46-909b-4b35-8925-fcdb98f5ee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 200 len: 1371 microversion: 2.20 time: 0.162475 Oct 28 11:07:30.313623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 333/655] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:30 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 1371 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:31.339483 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:31.340201 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:31.347109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:31.347584 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:31.348007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:31.348674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:31.349117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:31.349531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:31.363001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] HTTP exception thrown: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Request method failure captured: Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a HTTP/1.1 Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aHsv-Cx86mKuAI5Shg18AlPR6xZ9sSG68gjEeMNN4QBBuvVubjojiM-maRWCHJA1I-qtZUJi4kK3Mvi4JLVnFqpkrm_amShPb9gVMOrBf0R7ZOATcoqPjow565XoI6Yc4KKY4UR7UlVW9l1xhNOeSpJyMZIN6xBsGQYwhEPyYJIPI Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:07:31.364564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_312745d2566343faa50fa7c6f06483f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: fc81231c965947ecaced8013076edafc Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:07:31.366216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'e4ef9cf5-ad72-4399-a910-9308a243a57a'} Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:07:31.367124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:07:31.368597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:07:31.369942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Returning 404 to user: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:07:31.369942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4629fb9b-3fcf-4bf9-88e0-51cc0dca010d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 404 len: 111 microversion: 2.20 time: 0.030710 Oct 28 11:07:31.369942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 323/656] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:31 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:07:31.374809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-aed86748-bf9b-4990-bc76-7d462cfcec4b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '.version_select at 0x7f42fa5f0550>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:31.394699 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-aed86748-bf9b-4990-bc76-7d462cfcec4b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2010242548" status: 204 len: 0 microversion: 2.20 time: 0.023694 Oct 28 11:07:31.395264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 334/657] 10.222.0.46 () {60 vars in 1262 bytes} [Fri Oct 28 11:07:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2010242548 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 28 11:07:32.024017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d9c98db0-e601-406c-bbd6-8dd2621e3ad4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'create', calling method: .version_select at 0x7f42fa67e550>, body: {"keypair": {"name": "tempest-keypair-235302277", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA+Syi959PjuZXKAlOSBblXGaH5rWwudiQ0n6Do41lytNs1VZjiIG25sGtMx6RkoKHToElbsZVXroIqhz9+UadtHBexwfgDceMyA+J7HBD13uANQx9Mjvbjd4NWyqxGyhA=="}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:32.039468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-d9c98db0-e601-406c-bbd6-8dd2621e3ad4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for project 312745d2566343faa50fa7c6f06483f1. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:07:32.043295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-d9c98db0-e601-406c-bbd6-8dd2621e3ad4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for user fc81231c965947ecaced8013076edafc and project 312745d2566343faa50fa7c6f06483f1. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:07:32.068633 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d9c98db0-e601-406c-bbd6-8dd2621e3ad4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.048484 Oct 28 11:07:32.069147 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 324/658] 10.222.0.46 () {62 vars in 1200 bytes} [Fri Oct 28 11:07:32 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 50 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 28 11:07:33.275266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-599765754", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "95b80201-612c-4027-9944-5fd26540f482"}], "security_groups": [{"name": "tempest-securitygroup--709142987"}], "key_name": "tempest-keypair-235302277", "user_data": "}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:33.374186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] validate_networks() for [('95b80201-612c-4027-9944-5fd26540f482', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:07:33.783726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:07:33.784151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:07:33.784621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:07:33.785024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:07:33.785430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:07:33.794168 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:33.794647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:33.795078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:33.795793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:33.796240 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:33.796657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:33.813805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for project 312745d2566343faa50fa7c6f06483f1. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:07:33.817708 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Getting quotas for user fc81231c965947ecaced8013076edafc and project 312745d2566343faa50fa7c6f06483f1. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:07:33.829036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:07:33.831328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:07:33.877330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-540ca80d-f9e0-416e-bab2-62e4dc4c6c76 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.606328 Oct 28 11:07:33.877965 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 335/659] 10.222.0.46 () {62 vars in 1188 bytes} [Fri Oct 28 11:07:33 2022] POST /compute/v2.1/servers => generated 401 bytes in 608 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 28 11:07:33.889645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f2d7dd2b-a00e-492b-96cd-b7a36970d742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:33.890312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f2d7dd2b-a00e-492b-96cd-b7a36970d742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:33.948502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f2d7dd2b-a00e-492b-96cd-b7a36970d742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:33.955962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f2d7dd2b-a00e-492b-96cd-b7a36970d742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:33.957410 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f2d7dd2b-a00e-492b-96cd-b7a36970d742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:33.958743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f2d7dd2b-a00e-492b-96cd-b7a36970d742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1280 microversion: 2.20 time: 0.073708 Oct 28 11:07:33.959302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 325/660] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:33 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1280 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:34.985067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:34.985615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:34.991927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:34.992580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:34.993169 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:34.993847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:34.994480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:34.995653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:35.068536 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:35.076845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:35.078964 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:35.080869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a8bd6cc1-0b57-4960-853e-15cace9f7227 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1340 microversion: 2.20 time: 0.100729 Oct 28 11:07:35.081595 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 336/661] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:34 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1340 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:36.107758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:36.108548 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:36.115250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:36.115759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:36.116191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:36.116879 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:36.117359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:36.117757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:36.248672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:36.257993 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:36.259497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:36.260606 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a18bd94c-7e5d-4a83-a514-469c03564086 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.156810 Oct 28 11:07:36.261171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 326/662] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:36 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:36.630101 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-05b6fa60-1304-441b-8556-ce2e1bcc8197 req-013c66e0-1c00-40a2-b239-6bde20d36ec0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "057e4fde-a7eb-419f-b0ba-ad3e183bf191", "tag": "ded90a4e-c044-4824-8ed3-a7277034c619"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:36.639696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-05b6fa60-1304-441b-8556-ce2e1bcc8197 req-013c66e0-1c00-40a2-b239-6bde20d36ec0 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:36.640169 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-05b6fa60-1304-441b-8556-ce2e1bcc8197 req-013c66e0-1c00-40a2-b239-6bde20d36ec0 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:36.640633 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-05b6fa60-1304-441b-8556-ce2e1bcc8197 req-013c66e0-1c00-40a2-b239-6bde20d36ec0 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:36.652467 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-05b6fa60-1304-441b-8556-ce2e1bcc8197 req-013c66e0-1c00-40a2-b239-6bde20d36ec0 service nova] Creating event network-changed:ded90a4e-c044-4824-8ed3-a7277034c619 for instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 on hv-cinder-zed-3 Oct 28 11:07:36.657309 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-05b6fa60-1304-441b-8556-ce2e1bcc8197 req-013c66e0-1c00-40a2-b239-6bde20d36ec0 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031221 Oct 28 11:07:36.657901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 337/663] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:37.286376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:37.286981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:37.293312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:37.293779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:37.294250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:37.294924 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:37.295371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:37.295787 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:37.421018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:37.429499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:37.430964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:37.432056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-54abbf6e-6086-4371-a1fc-0639c2b8bb97 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.149605 Oct 28 11:07:37.432610 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 327/664] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:37 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:38.458776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:38.459393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:38.465509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:38.465999 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:38.466431 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:38.467101 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:38.467545 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:38.467966 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:38.592159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:38.600084 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:38.601511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:38.602639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d8be914b-b21a-4cbb-8840-c0de89792c0d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.148200 Oct 28 11:07:38.603186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 338/665] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:38 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:39.626660 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:39.627553 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:39.633668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:39.634188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:39.634615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:39.635472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:39.635805 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:39.636221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:39.769425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:39.777614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:39.779000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:39.780095 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b212ef1c-0a5d-4556-855a-ec0d79daec13 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.157328 Oct 28 11:07:39.780648 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 328/666] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:39 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:40.804815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:40.805348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:40.811240 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:40.811711 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:40.812140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:40.813316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:40.813761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:40.814202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:40.943927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:40.951950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:40.953389 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:40.954509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6bd40f27-92c5-4079-a7c7-92f649504b31 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.153624 Oct 28 11:07:40.955093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 339/667] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:40 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:41.979310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:41.979919 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:41.986561 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:41.987030 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:41.987455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:41.988127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:41.988569 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:41.988984 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:42.115568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:42.123530 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:42.125479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:42.126696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4b4b4cca-7345-43d6-beb5-bd64f88be172 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.151345 Oct 28 11:07:42.127231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 329/668] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:41 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:43.151280 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:43.151803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:43.158278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:43.158756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:43.159188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:43.159872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:43.160323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:43.160748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:43.288420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:43.296382 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:43.297841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:43.298957 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-361ca2ff-9205-46c0-ab71-cfd26137bee6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.151633 Oct 28 11:07:43.299507 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 340/669] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:43 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:43.717628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-a125f85b-c841-4a6f-871e-4e2731dee5f5 req-9bfcb16b-857c-436f-880a-ecfec69581ab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "057e4fde-a7eb-419f-b0ba-ad3e183bf191", "name": "network-vif-plugged", "status": "completed", "tag": "ded90a4e-c044-4824-8ed3-a7277034c619"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:43.729035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a125f85b-c841-4a6f-871e-4e2731dee5f5 req-9bfcb16b-857c-436f-880a-ecfec69581ab service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:43.729540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a125f85b-c841-4a6f-871e-4e2731dee5f5 req-9bfcb16b-857c-436f-880a-ecfec69581ab service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:43.730168 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a125f85b-c841-4a6f-871e-4e2731dee5f5 req-9bfcb16b-857c-436f-880a-ecfec69581ab service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:43.741285 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-a125f85b-c841-4a6f-871e-4e2731dee5f5 req-9bfcb16b-857c-436f-880a-ecfec69581ab service nova] Creating event network-vif-plugged:ded90a4e-c044-4824-8ed3-a7277034c619 for instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 on hv-cinder-zed-3 Oct 28 11:07:43.746388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-a125f85b-c841-4a6f-871e-4e2731dee5f5 req-9bfcb16b-857c-436f-880a-ecfec69581ab service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032332 Oct 28 11:07:43.746978 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 330/670] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:44.323807 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:44.324418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:44.326373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:44.327200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=6 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:44.333778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:44.334285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:44.334721 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:44.335408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:44.335860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:44.336284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:44.458876 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:44.466881 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:44.468328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:44.469466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9b0c178e-8e5e-40ef-aad1-96985e0160b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1405 microversion: 2.20 time: 0.149527 Oct 28 11:07:44.470040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 341/671] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:44 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1405 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:45.495244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:45.495845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:45.497794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:45.503773 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:45.504225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:45.504656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:45.505340 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:45.505790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:45.506238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:45.636352 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:45.645926 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:45.647440 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:45.648619 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e6d8d18a-71f6-4485-940f-a259c50ba360 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1597 microversion: 2.20 time: 0.158301 Oct 28 11:07:45.649177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 331/672] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:07:45 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1597 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:07:45.673393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:45.673885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:45.679740 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:45.680205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:45.680628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:45.681295 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:45.681739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:45.682197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:45.757139 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-da69fd8a-702d-4b1c-a586-d2c0455016f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.087605 Oct 28 11:07:45.757731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 342/673] 10.222.0.46 () {60 vars in 1316 bytes} [Fri Oct 28 11:07:45 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-interface => generated 285 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 11:07:46.654468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-28db0b02-ca5e-4d18-a8e1-c7f8d63ee29a req-36b8c993-ad35-4170-a49d-5d4e1b3711d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "057e4fde-a7eb-419f-b0ba-ad3e183bf191", "tag": "ded90a4e-c044-4824-8ed3-a7277034c619"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:46.664364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-28db0b02-ca5e-4d18-a8e1-c7f8d63ee29a req-36b8c993-ad35-4170-a49d-5d4e1b3711d2 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:46.664950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-28db0b02-ca5e-4d18-a8e1-c7f8d63ee29a req-36b8c993-ad35-4170-a49d-5d4e1b3711d2 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:46.665445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-28db0b02-ca5e-4d18-a8e1-c7f8d63ee29a req-36b8c993-ad35-4170-a49d-5d4e1b3711d2 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:46.676260 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-28db0b02-ca5e-4d18-a8e1-c7f8d63ee29a req-36b8c993-ad35-4170-a49d-5d4e1b3711d2 service nova] Creating event network-changed:ded90a4e-c044-4824-8ed3-a7277034c619 for instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 on hv-cinder-zed-3 Oct 28 11:07:46.681429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-28db0b02-ca5e-4d18-a8e1-c7f8d63ee29a req-36b8c993-ad35-4170-a49d-5d4e1b3711d2 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030642 Oct 28 11:07:46.682146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 332/674] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:48.486402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:48.486889 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:48.492649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:48.493125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:48.493554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:48.494256 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:48.494706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:48.495126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:48.573294 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lazy-loading 'services' on Instance uuid 01a19441-142f-41aa-9253-c5b50c929e93 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:07:48.645133 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6956b709-f537-4a48-97cf-0e4e7349a3db tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "DELETE /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments/24b8315c-e384-4a7f-b0ad-0c6f9ad8e79c" status: 202 len: 0 microversion: 2.1 time: 0.162722 Oct 28 11:07:48.645832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 343/675] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:07:48 2022] DELETE /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments/24b8315c-e384-4a7f-b0ad-0c6f9ad8e79c => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:07:48.655317 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:48.658386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:48.665319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:48.665884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:48.666481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:48.667193 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:48.667645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:48.668083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:48.699683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-cf276f07-8687-48ea-ab0c-b3d72d7427a9 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049249 Oct 28 11:07:48.700237 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 333/676] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:07:48 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:49.713307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:49.715613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:49.722351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:49.722822 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:49.723248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:49.723920 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:49.724362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:49.724774 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:49.753038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e209067a-d598-462f-afe8-d4bae9540c39 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044002 Oct 28 11:07:49.753578 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 344/677] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:07:49 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:07:49.828257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:49.828775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:49.835040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:49.835517 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:49.835948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:49.836623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:49.837071 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:49.837489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:49.863363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:07:49.935645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-575b173e-efd3-4865-82ab-ce70954606d8 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "DELETE /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 204 len: 0 microversion: 2.1 time: 0.110974 Oct 28 11:07:49.936135 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 334/678] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:07:49 2022] DELETE /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:07:49.944146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:49.944759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:49.950819 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:49.951297 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:49.951727 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:49.952402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:49.953384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:49.953800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:50.081590 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:50.089364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5db3d879-d181-4a78-948e-925b85dec525 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1689 microversion: 2.1 time: 0.148968 Oct 28 11:07:50.090232 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 345/679] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:49 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1689 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:51.113829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:51.114551 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:51.120838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:51.121316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:51.121748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:51.122460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:51.122911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:51.123332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:51.249460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:51.257699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d58d46cb-4165-480f-ab49-3d71f5f7aef2 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1689 microversion: 2.1 time: 0.148433 Oct 28 11:07:51.258285 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 335/680] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:51 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1689 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:52.280705 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:52.281915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:52.287895 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:52.288379 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:52.288811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:52.289493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:52.289949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:52.290402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:52.418097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:52.425724 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-343e693a-c7ea-4cf3-823e-3590a394704e tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1689 microversion: 2.1 time: 0.148932 Oct 28 11:07:52.426322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 346/681] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:52 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1689 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:07:53.447853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:53.448473 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:53.455492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:53.455970 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:53.456403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:53.457086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:53.457555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:53.457961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:53.460564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:53.586707 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:53.594397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-110323ef-8f3d-4f2e-a623-491401c0544b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1689 microversion: 2.1 time: 0.150314 Oct 28 11:07:53.595762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 336/682] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:53 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1689 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:54.281288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-52d5bf3f-b280-48ba-a01c-99ad801b8b2d req-e3885a4c-ecfc-475e-9bd0-5beda72a895a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01a19441-142f-41aa-9253-c5b50c929e93", "name": "network-vif-deleted", "tag": "eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:07:54.291973 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-52d5bf3f-b280-48ba-a01c-99ad801b8b2d req-e3885a4c-ecfc-475e-9bd0-5beda72a895a service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:54.292751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-52d5bf3f-b280-48ba-a01c-99ad801b8b2d req-e3885a4c-ecfc-475e-9bd0-5beda72a895a service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:54.293187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-52d5bf3f-b280-48ba-a01c-99ad801b8b2d req-e3885a4c-ecfc-475e-9bd0-5beda72a895a service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:54.305543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-52d5bf3f-b280-48ba-a01c-99ad801b8b2d req-e3885a4c-ecfc-475e-9bd0-5beda72a895a service nova] Creating event network-vif-deleted:eb53fbdf-f6b4-49e0-b7ed-08b48048ce0f for instance 01a19441-142f-41aa-9253-c5b50c929e93 on hv-cinder-zed-3 Oct 28 11:07:54.310882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-52d5bf3f-b280-48ba-a01c-99ad801b8b2d req-e3885a4c-ecfc-475e-9bd0-5beda72a895a service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033238 Oct 28 11:07:54.311780 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 347/683] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:07:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:07:54.618768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:54.619294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:54.621584 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:54.626742 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:54.627225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:54.627664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:54.628358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:54.628814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:54.629237 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:54.661648 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:54.662072 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:54.662512 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:54.662957 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lazy-loading 'fault' on Instance uuid 01a19441-142f-41aa-9253-c5b50c929e93 {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:07:54.705854 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:54.713691 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f313411f-5307-4946-bd27-427218bd595a tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 200 len: 1335 microversion: 2.1 time: 0.098840 Oct 28 11:07:54.714295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 337/684] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:54 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 1335 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:55.737391 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:55.738119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 01a19441-142f-41aa-9253-c5b50c929e93] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:55.740185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:55.746511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:55.746988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:55.747415 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:55.748090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:55.748535 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:55.748949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:55.763854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] HTTP exception thrown: Instance 01a19441-142f-41aa-9253-c5b50c929e93 could not be found. Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Request method failure captured: Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 HTTP/1.1 Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7aIhPtnefCCqNmAPadWpk5584o2_2r0isTmVkRGKzWI-I7c1QbiFIdoruXOy5h0VZogX9L4sPi-DVMWkisxvUM6avn2ESWp6fMy2fmzi6DazAAqVi14U8Y2drfllWP-AiHWNP38WDQIVdY8aOS1D9QC35crpyrYbxvM8s51pB0LFgc Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:07:55.765646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 347e262907da4a83a197aaf42cecf51d Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 01a19441-142f-41aa-9253-c5b50c929e93 could not be found. Oct 28 11:07:55.767723 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '01a19441-142f-41aa-9253-c5b50c929e93'} Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01a19441-142f-41aa-9253-c5b50c929e93 could not be found. Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:07:55.768488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01a19441-142f-41aa-9253-c5b50c929e93 could not be found. Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01a19441-142f-41aa-9253-c5b50c929e93 could not be found. Oct 28 11:07:55.770067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:07:55.771482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Returning 404 to user: Instance 01a19441-142f-41aa-9253-c5b50c929e93 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:07:55.771482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-689f37ba-4a04-4996-9260-12395bec635f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93" status: 404 len: 111 microversion: 2.1 time: 0.034710 Oct 28 11:07:55.771482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 348/685] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:55 2022] GET /compute/v2.1/servers/01a19441-142f-41aa-9253-c5b50c929e93 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:07:55.776592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-99bbf92c-938f-4914-8186-50bc4f1efb6f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '.version_select at 0x7f42fa670280>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:55.795859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-99bbf92c-938f-4914-8186-50bc4f1efb6f tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-189909379" status: 202 len: 0 microversion: 2.1 time: 0.023063 Oct 28 11:07:55.796470 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 338/686] 10.222.0.46 () {58 vars in 1218 bytes} [Fri Oct 28 11:07:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-189909379 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:07:58.641677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:58.642249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:58.648050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:58.648526 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:58.648963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:58.649642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:58.650129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:58.650554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:58.653046 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=89,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:07:58.677499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:07:58.751157 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ecb75654-359d-4b03-84d5-be5df2e3c129 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "DELETE /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 204 len: 0 microversion: 2.1 time: 0.112918 Oct 28 11:07:58.751288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 349/687] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:07:58 2022] DELETE /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:07:58.760157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:58.760770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:58.767613 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:58.768095 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:58.768560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:58.769249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:58.769700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:58.770213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:58.894668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:07:58.902278 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d4594058-d0ae-4db1-8399-d19b1a2543f7 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1508 microversion: 2.1 time: 0.146634 Oct 28 11:07:58.902845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 339/688] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:58 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1508 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:07:59.927189 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:07:59.927885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:07:59.933726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:59.934239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:59.934670 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:07:59.935350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:07:59.935800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:07:59.936219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:00.062744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:00.071448 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4a61923e-99c4-4302-8130-cc4757bb3001 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1508 microversion: 2.1 time: 0.149124 Oct 28 11:08:00.072041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 350/689] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:07:59 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1508 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:00.745798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-34e98771-8848-4bbe-9e9c-9a89c3b6969d req-9bf6b445-e190-4b4e-9a58-de1e15c09f03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "643427ec-cf7f-4058-a731-b96412826c2e", "name": "network-vif-deleted", "tag": "c012cb5d-de75-4d0c-8ffe-7a84c1ef9d66"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:00.756502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-34e98771-8848-4bbe-9e9c-9a89c3b6969d req-9bf6b445-e190-4b4e-9a58-de1e15c09f03 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:00.757061 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-34e98771-8848-4bbe-9e9c-9a89c3b6969d req-9bf6b445-e190-4b4e-9a58-de1e15c09f03 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:00.757390 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-34e98771-8848-4bbe-9e9c-9a89c3b6969d req-9bf6b445-e190-4b4e-9a58-de1e15c09f03 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:00.771999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-34e98771-8848-4bbe-9e9c-9a89c3b6969d req-9bf6b445-e190-4b4e-9a58-de1e15c09f03 service nova] Creating event network-vif-deleted:c012cb5d-de75-4d0c-8ffe-7a84c1ef9d66 for instance 643427ec-cf7f-4058-a731-b96412826c2e on hv-cinder-zed-3 Oct 28 11:08:00.777109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-34e98771-8848-4bbe-9e9c-9a89c3b6969d req-9bf6b445-e190-4b4e-9a58-de1e15c09f03 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034904 Oct 28 11:08:00.777671 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 340/690] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:01.093143 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:01.093762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:01.100714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:01.101211 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:01.101654 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:01.102504 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:01.102960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:01.103386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:01.136544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:01.137024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:01.137460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:01.137925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lazy-loading 'fault' on Instance uuid 643427ec-cf7f-4058-a731-b96412826c2e {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:08:01.179251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:01.187072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-32b14fd5-a5ea-4fb2-8914-ea34efc2b3dd tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 200 len: 1312 microversion: 2.1 time: 0.097843 Oct 28 11:08:01.187638 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 351/691] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:01 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 1312 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:02.210979 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:02.211677 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 643427ec-cf7f-4058-a731-b96412826c2e] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:02.218020 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:02.218504 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:02.218969 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:02.219654 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:02.220112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:02.220528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:02.233497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] HTTP exception thrown: Instance 643427ec-cf7f-4058-a731-b96412826c2e could not be found. Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Request method failure captured: Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e HTTP/1.1 Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aIhPtnefCCqNmAPadWpk5584o2_2r0isTmVkRGKzWI-I7c1QbiFIdoruXOy5h0VZogX9L4sPi-DVMWkisxvUM6avn2ESWp6fMy2fmzi6DazAAqVi14U8Y2drfllWP-AiHWNP38WDQIVdY8aOS1D9QC35crpyrYbxvM8s51pB0LFgc Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:08:02.235126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 347e262907da4a83a197aaf42cecf51d Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 643427ec-cf7f-4058-a731-b96412826c2e could not be found. Oct 28 11:08:02.237328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '643427ec-cf7f-4058-a731-b96412826c2e'} Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 643427ec-cf7f-4058-a731-b96412826c2e could not be found. Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:08:02.238099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 643427ec-cf7f-4058-a731-b96412826c2e could not be found. Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 643427ec-cf7f-4058-a731-b96412826c2e could not be found. Oct 28 11:08:02.239446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:02.240797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Returning 404 to user: Instance 643427ec-cf7f-4058-a731-b96412826c2e could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:08:02.240797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6573090b-c051-4046-9168-34d884e36efe tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e" status: 404 len: 111 microversion: 2.1 time: 0.031013 Oct 28 11:08:02.240797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 341/692] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:02 2022] GET /compute/v2.1/servers/643427ec-cf7f-4058-a731-b96412826c2e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:08:02.245813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:02.246345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:02.252988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:02.253480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:02.253917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:02.254771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:02.255099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:02.255527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:02.281120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:08:02.354923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d2757b12-501f-47c7-837a-11311a6f050b tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "DELETE /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 204 len: 0 microversion: 2.1 time: 0.112627 Oct 28 11:08:02.355423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 352/693] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:08:02 2022] DELETE /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:08:02.364444 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:02.365059 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:02.371074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:02.371550 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:02.371981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:02.372667 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:02.373118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:02.373536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:02.503083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:02.510725 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-627a794e-1407-48b9-a247-374b7b571ff4 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.151010 Oct 28 11:08:02.511283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 342/694] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:02 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:03.533252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:03.533869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:03.539719 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:03.540202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:03.540636 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:03.541312 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:03.541761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:03.542219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:03.676287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:03.683933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-da7e4547-aeb7-4c9e-8f4f-350c921ffb44 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.155032 Oct 28 11:08:03.684496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 353/695] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:03 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:04.708060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:04.708688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:04.715117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:04.715588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:04.716014 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:04.716687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:04.717134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:04.717549 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:04.842513 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:04.850230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-aa6f7adc-1934-4b49-af98-9d4bccec7135 tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1522 microversion: 2.1 time: 0.146605 Oct 28 11:08:04.850796 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 343/696] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:04 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1522 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:05.550485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-0472bdc0-771c-48b3-a231-374128990a95 req-c95ade51-7b0a-492e-8aa9-38bd882c7ba7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54cf39f2-cfd4-4f12-bddb-36666db9e1d5", "name": "network-vif-deleted", "tag": "9cdeceb0-c9f5-42e0-9e0e-62d116b04c2f"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:05.561550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-0472bdc0-771c-48b3-a231-374128990a95 req-c95ade51-7b0a-492e-8aa9-38bd882c7ba7 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:05.562178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-0472bdc0-771c-48b3-a231-374128990a95 req-c95ade51-7b0a-492e-8aa9-38bd882c7ba7 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:05.562681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-0472bdc0-771c-48b3-a231-374128990a95 req-c95ade51-7b0a-492e-8aa9-38bd882c7ba7 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:05.575999 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-0472bdc0-771c-48b3-a231-374128990a95 req-c95ade51-7b0a-492e-8aa9-38bd882c7ba7 service nova] Creating event network-vif-deleted:9cdeceb0-c9f5-42e0-9e0e-62d116b04c2f for instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 on hv-cinder-zed-3 Oct 28 11:08:05.583152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-0472bdc0-771c-48b3-a231-374128990a95 req-c95ade51-7b0a-492e-8aa9-38bd882c7ba7 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036799 Oct 28 11:08:05.584094 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 354/697] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:05.872491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:05.873037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:05.879406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:05.879890 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:05.880329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:05.881007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:05.881457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:05.881877 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:05.912372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:05.912840 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:05.913274 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:05.913711 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lazy-loading 'fault' on Instance uuid 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:08:05.952043 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '347e262907da4a83a197aaf42cecf51d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f49235fbe64b79bfef2cf828993c56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:05.960039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f61be5aa-f3a6-4af2-ba88-10edd8af32aa tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 200 len: 1312 microversion: 2.1 time: 0.091432 Oct 28 11:08:05.960593 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 344/698] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:05 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 1312 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:06.983176 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:06.983873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] [instance: 54cf39f2-cfd4-4f12-bddb-36666db9e1d5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:06.990363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:06.990846 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:06.991284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:06.991970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:06.992433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:06.992891 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:07.006333 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] HTTP exception thrown: Instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 could not be found. Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Request method failure captured: Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 HTTP/1.1 Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7aIhPtnefCCqNmAPadWpk5584o2_2r0isTmVkRGKzWI-I7c1QbiFIdoruXOy5h0VZogX9L4sPi-DVMWkisxvUM6avn2ESWp6fMy2fmzi6DazAAqVi14U8Y2drfllWP-AiHWNP38WDQIVdY8aOS1D9QC35crpyrYbxvM8s51pB0LFgc Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:08:07.007911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/f1f49235fbe64b79bfef2cf828993c56"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: f1f49235fbe64b79bfef2cf828993c56 Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1937195262 Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 347e262907da4a83a197aaf42cecf51d Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1937195262-project Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 could not be found. Oct 28 11:08:07.009505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '54cf39f2-cfd4-4f12-bddb-36666db9e1d5'} Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 could not be found. Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:08:07.010439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 could not be found. Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 could not be found. Oct 28 11:08:07.011988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:07.013348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] Returning 404 to user: Instance 54cf39f2-cfd4-4f12-bddb-36666db9e1d5 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:08:07.013348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f22735e7-3f9f-424e-b898-ca4fd43282eb tempest-ServerRescueNegativeTestJSON-1937195262 tempest-ServerRescueNegativeTestJSON-1937195262-project] 10.222.0.46 "GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5" status: 404 len: 111 microversion: 2.1 time: 0.030313 Oct 28 11:08:07.013348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 355/699] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:06 2022] GET /compute/v2.1/servers/54cf39f2-cfd4-4f12-bddb-36666db9e1d5 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:08:15.410906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8a9706e0-4b91-47b4-a1ec-7004bcbebe36 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'create', calling method: .version_select at 0x7f42fa633af0>, body: {"keypair": {"name": "tempest-keypair-804514549", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE609vu7Zb9NeoAk7frHuQ0vjk87zauqwK9VqSKHZTT7l3LfqTOAtshyRwT2jLzbfPZpFz0gPqPd6Zlzj2LSs51ag2x4Mi/AYA45I4T0QT6EVbi2GILZqgWJA6RpktCsbw=="}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:15.428920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-8a9706e0-4b91-47b4-a1ec-7004bcbebe36 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for project 64177183598843a68fc7fbe8257d4881. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:08:15.432777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-8a9706e0-4b91-47b4-a1ec-7004bcbebe36 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for user a1f71865b51b4e8480932fffae985567 and project 64177183598843a68fc7fbe8257d4881. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:08:15.457594 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8a9706e0-4b91-47b4-a1ec-7004bcbebe36 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.177048 Oct 28 11:08:15.458412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 345/700] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:08:15 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:16.788224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1353840739", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6b052aa7-a953-4b1a-862c-d4362c935620"}], "security_groups": [{"name": "tempest-securitygroup--630464286"}], "key_name": "tempest-keypair-804514549", "user_data": "}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:16.827492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=43 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:16.831961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1420206025"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:16.832486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:16.839438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:16.839914 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:16.840337 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:16.841017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:16.841462 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:16.841880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:16.896069 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] validate_networks() for [('6b052aa7-a953-4b1a-862c-d4362c935620', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:08:16.985117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.network.security_group_api [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Removing security group dea451e8-7965-49ca-9e86-e29c4b0c3b6e from port 6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 Oct 28 11:08:16.998229 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=51,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:17.292122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:08:17.292622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:08:17.293321 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:08:17.293969 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:08:17.294583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:08:17.304882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:17.305680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:17.306162 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:17.306901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:17.307362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:17.307791 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:17.327062 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for project 64177183598843a68fc7fbe8257d4881. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:08:17.332043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for user a1f71865b51b4e8480932fffae985567 and project 64177183598843a68fc7fbe8257d4881. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:08:17.344471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:08:17.347108 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:08:17.399315 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-92a12ee8-d813-42f4-8882-aa5b45da9cea tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.615337 Oct 28 11:08:17.405885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 356/701] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:08:16 2022] POST /compute/v2.1/servers => generated 401 bytes in 617 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:08:17.411507 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2d274fb7-284f-4cda-a638-c40c02fbb88f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:17.412233 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2d274fb7-284f-4cda-a638-c40c02fbb88f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:17.474909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2d274fb7-284f-4cda-a638-c40c02fbb88f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:17.484767 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2d274fb7-284f-4cda-a638-c40c02fbb88f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1237 microversion: 2.1 time: 0.077459 Oct 28 11:08:17.485830 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 357/702] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:17 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1237 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:17.595415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-eb4157a1-4763-40f0-a858-a2ba1020a156 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/action" status: 202 len: 0 microversion: 2.1 time: 0.768671 Oct 28 11:08:17.597296 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 346/703] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:08:16 2022] POST /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/action => generated 0 bytes in 771 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 11:08:17.660202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-dbbbf80c-9e2e-4e6b-92c3-5830c2ae4a48 req-16ed9cfd-2cd9-4340-8a4b-9083f32c12ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "818ad5c3-a0f1-437e-9fe3-45242c1f7bd3", "tag": "6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:17.678746 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-dbbbf80c-9e2e-4e6b-92c3-5830c2ae4a48 req-16ed9cfd-2cd9-4340-8a4b-9083f32c12ad service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:17.679570 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-dbbbf80c-9e2e-4e6b-92c3-5830c2ae4a48 req-16ed9cfd-2cd9-4340-8a4b-9083f32c12ad service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:17.680523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-dbbbf80c-9e2e-4e6b-92c3-5830c2ae4a48 req-16ed9cfd-2cd9-4340-8a4b-9083f32c12ad service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:17.712081 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-dbbbf80c-9e2e-4e6b-92c3-5830c2ae4a48 req-16ed9cfd-2cd9-4340-8a4b-9083f32c12ad service nova] Creating event network-changed:6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 for instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 on hv-cinder-zed-3 Oct 28 11:08:17.724378 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-dbbbf80c-9e2e-4e6b-92c3-5830c2ae4a48 req-16ed9cfd-2cd9-4340-8a4b-9083f32c12ad service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068444 Oct 28 11:08:17.728851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 358/704] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:18.510439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:18.511085 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:18.518527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:18.519087 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:18.519529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:18.520230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:18.520857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:18.521459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:18.593525 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:18.602776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d08ab59c-e32a-4b50-ab0d-ee156721d007 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1289 microversion: 2.1 time: 0.096345 Oct 28 11:08:18.603383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 347/705] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:18 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1289 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:19.117368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:19.118119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:19.125264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:19.126746 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:19.127371 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:19.128250 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:19.128857 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:19.129435 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:19.233851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lazy-loading 'services' on Instance uuid 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:08:19.328020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6f5181f8-be79-42a2-ad7f-2c1c0e12b7eb tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "DELETE /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments/7441ee77-9c3f-487a-9d51-7462f1919efa" status: 202 len: 0 microversion: 2.1 time: 0.214992 Oct 28 11:08:19.332719 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 359/706] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:08:19 2022] DELETE /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments/7441ee77-9c3f-487a-9d51-7462f1919efa => generated 0 bytes in 221 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:08:19.339000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:19.341433 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:19.358104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:19.358687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:19.359351 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:19.361684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:19.362604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:19.363221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:19.407689 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b827b8d4-2084-42b6-8aa0-70ebdcbb8761 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073734 Oct 28 11:08:19.408428 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 348/707] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:08:19 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:19.628426 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:19.629199 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:19.644472 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:19.647934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:19.648658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:19.649788 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:19.650328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:19.650752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:19.748295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-6613fbaf-04ab-499b-aa05-e11304b5dba0 req-3b5b1743-78d7-4017-a975-2fa7d982daea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "818ad5c3-a0f1-437e-9fe3-45242c1f7bd3", "tag": "6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:19.760553 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-6613fbaf-04ab-499b-aa05-e11304b5dba0 req-3b5b1743-78d7-4017-a975-2fa7d982daea service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:19.761127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-6613fbaf-04ab-499b-aa05-e11304b5dba0 req-3b5b1743-78d7-4017-a975-2fa7d982daea service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:19.761756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-6613fbaf-04ab-499b-aa05-e11304b5dba0 req-3b5b1743-78d7-4017-a975-2fa7d982daea service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:19.774748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-6613fbaf-04ab-499b-aa05-e11304b5dba0 req-3b5b1743-78d7-4017-a975-2fa7d982daea service nova] Creating event network-changed:6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 for instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 on hv-cinder-zed-3 Oct 28 11:08:19.782816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-6613fbaf-04ab-499b-aa05-e11304b5dba0 req-3b5b1743-78d7-4017-a975-2fa7d982daea service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040333 Oct 28 11:08:19.786144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 349/708] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:19.817656 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:19.825561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-617a6c15-c52f-4ee7-a2ca-c60602a40347 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.202592 Oct 28 11:08:19.826157 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 360/709] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:19 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:20.421387 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:20.424284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:20.436135 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:20.436895 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:20.437528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:20.438764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:20.439396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:20.440375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:20.473594 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b064e7b4-cc6c-45e6-8789-4cb809501067 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.056706 Oct 28 11:08:20.475332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 350/710] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:08:20 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments => generated 25 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:08:20.487007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:20.488143 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:20.495592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:20.496574 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:20.497429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:20.498768 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:20.499402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:20.499983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:20.614459 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] HTTP exception thrown: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 is not attached to volume 7441ee77-9c3f-487a-9d51-7462f1919efa Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Request method failure captured: Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments/7441ee77-9c3f-487a-9d51-7462f1919efa HTTP/1.1 Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7c7Qh0KUQPrzXgDoeH9tmTCIljRbzZPDpuTwqI-7n0Njo2A9OBDeA7qELN597kh3T62Q9bzhUBP0quwWs-_bmB_XIPSXcFnDvHO9cbay7rgmapARm6qjCoDjzZDaVcJvNHL2f4Sz_1Qs7jF0c5MQX-pp9fTmOTPl1ObsHuHD57tyok Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 27b47aa9a61b4898a9af0062896cc3da Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-TestMinimumBasicScenario-515944491 Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:08:20.622811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-TestMinimumBasicScenario-515944491 Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 27b47aa9a61b4898a9af0062896cc3da Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-TestMinimumBasicScenario-515944491 Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-TestMinimumBasicScenario-515944491-project Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 859315e3352843fd801ddc6568729e56 Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-TestMinimumBasicScenario-515944491-project Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:08:20.624402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 is not attached to volume 7441ee77-9c3f-487a-9d51-7462f1919efa Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'server_id': '818ad5c3-a0f1-437e-9fe3-45242c1f7bd3', 'id': '7441ee77-9c3f-487a-9d51-7462f1919efa'} Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 7441ee77-9c3f-487a-9d51-7462f1919efa. Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:08:20.625803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 is not attached to volume 7441ee77-9c3f-487a-9d51-7462f1919efa Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 is not attached to volume 7441ee77-9c3f-487a-9d51-7462f1919efa Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:08:20.627328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Returning 404 to user: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 is not attached to volume 7441ee77-9c3f-487a-9d51-7462f1919efa {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:08:20.628283 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2587031c-9f5e-4a35-921d-356a99596198 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "DELETE /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments/7441ee77-9c3f-487a-9d51-7462f1919efa" status: 404 len: 154 microversion: 2.1 time: 0.144699 Oct 28 11:08:20.628283 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 361/711] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:08:20 2022] DELETE /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3/os-volume_attachments/7441ee77-9c3f-487a-9d51-7462f1919efa => generated 154 bytes in 147 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:08:20.742008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-7be65785-a6a6-451b-ab91-40a5b5d41191 req-cc52b7b5-efa4-41bd-8a89-02e5c73db2b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4432177-31d4-43c8-a6a2-a1d727b144a2", "tag": "c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:20.752496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-7be65785-a6a6-451b-ab91-40a5b5d41191 req-cc52b7b5-efa4-41bd-8a89-02e5c73db2b8 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:20.753676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-7be65785-a6a6-451b-ab91-40a5b5d41191 req-cc52b7b5-efa4-41bd-8a89-02e5c73db2b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:20.754293 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-7be65785-a6a6-451b-ab91-40a5b5d41191 req-cc52b7b5-efa4-41bd-8a89-02e5c73db2b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:20.765513 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-7be65785-a6a6-451b-ab91-40a5b5d41191 req-cc52b7b5-efa4-41bd-8a89-02e5c73db2b8 service nova] Creating event network-changed:c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0 for instance e4432177-31d4-43c8-a6a2-a1d727b144a2 on hv-cinder-zed-3 Oct 28 11:08:20.770653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-7be65785-a6a6-451b-ab91-40a5b5d41191 req-cc52b7b5-efa4-41bd-8a89-02e5c73db2b8 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032326 Oct 28 11:08:20.771264 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 351/712] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:08:20.849425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:20.850192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:20.856033 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:20.856578 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:20.857072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:20.857820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:20.858366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:20.858867 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:21.070968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:21.078639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c5c7e18d-b19d-4432-9502-c14988b85717 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.233893 Oct 28 11:08:21.079825 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 362/713] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:20 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:21.780487 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:21.781070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:21.787603 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:21.788079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:21.788507 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:21.789187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:21.789637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:21.790091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:21.814481 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:08:21.885948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-bd36ab3e-87d5-41f0-80d5-c27be310b934 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "DELETE /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 204 len: 0 microversion: 2.1 time: 0.109762 Oct 28 11:08:21.886557 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 352/714] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:08:21 2022] DELETE /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:08:21.894552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:21.895163 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:21.900941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:21.901423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:21.901856 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:21.902578 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:21.903034 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:21.903474 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:22.030429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:22.038763 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-eca5323d-948e-4228-8bd2-c5a05cc5b413 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1553 microversion: 2.1 time: 0.147988 Oct 28 11:08:22.039337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 363/715] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:21 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1553 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:22.102186 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:22.102922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:22.108974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:22.109345 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:22.109851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:22.111150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:22.111855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:22.112270 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:22.242918 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:22.250739 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d9526c67-c5bd-47b9-9aac-1098e9473b72 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.152713 Oct 28 11:08:22.251462 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 353/716] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:22 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:23.062840 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:23.063499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:23.070048 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:23.070530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:23.070968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:23.071658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:23.072140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:23.072565 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:23.196933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:23.205083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a97ddb9e-2c9a-4adc-88d7-86d2df3148c2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1553 microversion: 2.1 time: 0.146149 Oct 28 11:08:23.205679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 364/717] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:23 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1553 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:23.273461 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:23.274126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:23.280403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:23.280878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:23.281308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:23.282052 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:23.282544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:23.282962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:23.415872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:23.424818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8cfa3a34-7234-440a-bf8b-10d4245ce628 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.155135 Oct 28 11:08:23.425495 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 354/718] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:23 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:24.228554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:24.229172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:24.235109 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:24.236155 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:24.236849 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:24.237539 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:24.238005 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:24.238434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:24.366366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:24.374003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2a4dd978-c39c-41ba-b90e-f28ebc7f8d4b tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1553 microversion: 2.1 time: 0.149252 Oct 28 11:08:24.374568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 365/719] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:24 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1553 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:24.447955 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:24.448562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:24.455479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:24.455959 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:24.456388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:24.457067 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:24.457513 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:24.457932 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:24.584709 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:24.592218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e782d4ae-5885-4c17-bdf3-f775db087ce5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.148170 Oct 28 11:08:24.592777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 355/720] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:24 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:25.396933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:25.398158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:25.404230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:25.404713 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:25.405151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:25.405862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:25.406345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:25.406762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:25.615162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:25.615881 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:25.618028 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=4 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:25.619139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=57,nova_api:DELETE=1,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:25.622799 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '859315e3352843fd801ddc6568729e56', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27b47aa9a61b4898a9af0062896cc3da', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:25.624115 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:25.624797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:25.625229 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:25.625919 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:25.626395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:25.626804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:25.637805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6eb5867b-16f6-4fbc-b927-42de91ededab tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 200 len: 1553 microversion: 2.1 time: 0.244644 Oct 28 11:08:25.655599 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 366/721] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:25 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 1553 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:25.762138 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:25.771110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7eb24476-69d5-4e89-ad6d-ab7e1c98d597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.160473 Oct 28 11:08:25.771737 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 356/722] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:25 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:25.847945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-d9a1f403-45a3-4bb5-85ca-97dae2fa527d req-808feee4-4f09-42e9-a1f3-e4ff17d91cf9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "818ad5c3-a0f1-437e-9fe3-45242c1f7bd3", "name": "network-vif-deleted", "tag": "6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:25.857581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d9a1f403-45a3-4bb5-85ca-97dae2fa527d req-808feee4-4f09-42e9-a1f3-e4ff17d91cf9 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:25.858118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d9a1f403-45a3-4bb5-85ca-97dae2fa527d req-808feee4-4f09-42e9-a1f3-e4ff17d91cf9 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:25.858542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d9a1f403-45a3-4bb5-85ca-97dae2fa527d req-808feee4-4f09-42e9-a1f3-e4ff17d91cf9 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:25.870901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-d9a1f403-45a3-4bb5-85ca-97dae2fa527d req-808feee4-4f09-42e9-a1f3-e4ff17d91cf9 service nova] Creating event network-vif-deleted:6e8dd8eb-d2ee-4e61-a65a-9b2f09d40151 for instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 on hv-cinder-zed-3 Oct 28 11:08:25.876299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-d9a1f403-45a3-4bb5-85ca-97dae2fa527d req-808feee4-4f09-42e9-a1f3-e4ff17d91cf9 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032404 Oct 28 11:08:25.879877 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 367/723] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:26.663380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:26.663986 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] [instance: 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:26.670313 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:26.670792 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:26.671219 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:26.671895 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:26.672342 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:26.672757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:26.685361 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] HTTP exception thrown: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 could not be found. Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Request method failure captured: Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 HTTP/1.1 Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7c7Qh0KUQPrzXgDoeH9tmTCIljRbzZPDpuTwqI-7n0Njo2A9OBDeA7qELN597kh3T62Q9bzhUBP0quwWs-_bmB_XIPSXcFnDvHO9cbay7rgmapARm6qjCoDjzZDaVcJvNHL2f4Sz_1Qs7jF0c5MQX-pp9fTmOTPl1ObsHuHD57tyok Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 27b47aa9a61b4898a9af0062896cc3da Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-TestMinimumBasicScenario-515944491 Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:08:26.686974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/27b47aa9a61b4898a9af0062896cc3da"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-TestMinimumBasicScenario-515944491 Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 27b47aa9a61b4898a9af0062896cc3da Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-TestMinimumBasicScenario-515944491 Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-TestMinimumBasicScenario-515944491-project Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 859315e3352843fd801ddc6568729e56 Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-TestMinimumBasicScenario-515944491-project Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 could not be found. Oct 28 11:08:26.688759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '818ad5c3-a0f1-437e-9fe3-45242c1f7bd3'} Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 could not be found. Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:08:26.690459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 could not be found. Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 could not be found. Oct 28 11:08:26.692091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:08:26.693562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Returning 404 to user: Instance 818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:08:26.693562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-58ec4a0f-3682-4854-974a-2a05e9908729 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3" status: 404 len: 111 microversion: 2.1 time: 0.028865 Oct 28 11:08:26.693562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 357/724] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:26 2022] GET /compute/v2.1/servers/818ad5c3-a0f1-437e-9fe3-45242c1f7bd3 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:08:26.697616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c6864fd8-dd7c-42d5-a6fd-9a7bd6e1c5f2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] Calling method '.version_select at 0x7f42fbe79940>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:26.716056 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c6864fd8-dd7c-42d5-a6fd-9a7bd6e1c5f2 tempest-TestMinimumBasicScenario-515944491 tempest-TestMinimumBasicScenario-515944491-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1518528129" status: 202 len: 0 microversion: 2.1 time: 0.022580 Oct 28 11:08:26.716630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 368/725] 10.222.0.46 () {58 vars in 1272 bytes} [Fri Oct 28 11:08:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1518528129 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:08:26.794297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:26.794908 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:26.800642 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:26.801116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:26.801547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:26.802252 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:26.802700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:26.803302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:26.933032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:26.940592 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3ab64483-d2b3-46f2-8ee2-7e5b8f2301a9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.150036 Oct 28 11:08:26.941160 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 358/726] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:26 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:27.951314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:27.952676 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=8 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:27.958591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-6686606b-caef-4d0f-9f53-5d28105a0b17 req-a1b7728f-cbfe-401e-94b3-8f95e16d43be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4432177-31d4-43c8-a6a2-a1d727b144a2", "name": "network-vif-plugged", "status": "completed", "tag": "c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:27.966759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:27.968285 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:27.974292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6686606b-caef-4d0f-9f53-5d28105a0b17 req-a1b7728f-cbfe-401e-94b3-8f95e16d43be service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:27.974995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6686606b-caef-4d0f-9f53-5d28105a0b17 req-a1b7728f-cbfe-401e-94b3-8f95e16d43be service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:27.976473 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6686606b-caef-4d0f-9f53-5d28105a0b17 req-a1b7728f-cbfe-401e-94b3-8f95e16d43be service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:27.976771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:27.977548 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:27.978206 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:27.979190 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:27.979872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:27.980718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:27.990004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-6686606b-caef-4d0f-9f53-5d28105a0b17 req-a1b7728f-cbfe-401e-94b3-8f95e16d43be service nova] Creating event network-vif-plugged:c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0 for instance e4432177-31d4-43c8-a6a2-a1d727b144a2 on hv-cinder-zed-3 Oct 28 11:08:27.995471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-6686606b-caef-4d0f-9f53-5d28105a0b17 req-a1b7728f-cbfe-401e-94b3-8f95e16d43be service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044878 Oct 28 11:08:27.996594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 369/727] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:28.147583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:28.155303 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-af5c7aed-b451-428a-85f3-eac203ce172b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.194273 Oct 28 11:08:28.155874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 359/728] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:27 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:29.179064 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:29.179764 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:29.181948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=2,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:29.189339 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:29.190036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:29.190511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:29.191220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:29.191684 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:29.192114 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:29.394640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:29.403473 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ac3dd572-bdfd-4000-85db-eb365da8b824 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1362 microversion: 2.1 time: 0.228498 Oct 28 11:08:29.407761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 370/729] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:29 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1362 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:30.426972 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:30.427790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:30.434490 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:30.435042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:30.435540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:30.436318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:30.436844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:30.437331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:30.651932 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:08:30.660841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-998876f1-d020-4837-91c9-d7f463830876 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1547 microversion: 2.1 time: 0.238434 Oct 28 11:08:30.661415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 360/730] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:08:30 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1547 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:08:30.683483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:08:30.684043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:08:30.689879 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:30.690385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:30.690817 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:30.691495 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:30.691972 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:30.692387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:30.772640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5a410349-bdbe-4c68-bb91-7c2c84bbf67c tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093171 Oct 28 11:08:30.773266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 371/731] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:08:30 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:31.978435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-e3faf5af-3237-4cfe-b496-dc180f162fd7 req-74495f63-55c9-4848-ad30-96fbdccd4561 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4432177-31d4-43c8-a6a2-a1d727b144a2", "tag": "c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:31.988953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e3faf5af-3237-4cfe-b496-dc180f162fd7 req-74495f63-55c9-4848-ad30-96fbdccd4561 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:08:31.989568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e3faf5af-3237-4cfe-b496-dc180f162fd7 req-74495f63-55c9-4848-ad30-96fbdccd4561 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:08:31.990137 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e3faf5af-3237-4cfe-b496-dc180f162fd7 req-74495f63-55c9-4848-ad30-96fbdccd4561 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:08:32.001503 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-e3faf5af-3237-4cfe-b496-dc180f162fd7 req-74495f63-55c9-4848-ad30-96fbdccd4561 service nova] Creating event network-changed:c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0 for instance e4432177-31d4-43c8-a6a2-a1d727b144a2 on hv-cinder-zed-3 Oct 28 11:08:32.006961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-e3faf5af-3237-4cfe-b496-dc180f162fd7 req-74495f63-55c9-4848-ad30-96fbdccd4561 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032043 Oct 28 11:08:32.011642 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 361/732] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:08:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:08:44.337964 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:44.339165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=69,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:08:44.585198 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ef8189dd-9ea8-44a1-a943-6329f43d4d82 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Action: 'create', calling method: .version_select at 0x7f42fa6150d0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1437936952", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHw+Kw9WOmlEuP2bRGgCmxkt7hnq7VDgeWKwSJWO3n08rktMPbM9ZB5FIDd6S8y0impc3bkQUuZIdysIfgC2RaYoOBt/ov/EiZQR///jKX58mJyeQc0gJh6Hpi0cd0EhnQ=="}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:08:44.602194 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-ef8189dd-9ea8-44a1-a943-6329f43d4d82 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Getting quotas for project 9edc5ff928624d87944ee071e578aa17. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:08:44.607281 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-ef8189dd-9ea8-44a1-a943-6329f43d4d82 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Getting quotas for user d4ade5932a324a4b8956b5377256ce7a and project 9edc5ff928624d87944ee071e578aa17. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:08:44.630813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ef8189dd-9ea8-44a1-a943-6329f43d4d82 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.293910 Oct 28 11:08:44.631419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 372/733] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:08:44 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:09:02.934390 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:02.934974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:02.937309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=11 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:02.937958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=86,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:02.944598 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:02.945079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:02.945514 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:02.946225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:02.946696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:02.947099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:02.971188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b8211309-c44d-4448-838b-f89ceaa3cc11 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/ips" status: 200 len: 151 microversion: 2.20 time: 0.041003 Oct 28 11:09:02.971756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 362/734] 10.222.0.46 () {60 vars in 1289 bytes} [Fri Oct 28 11:09:02 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/ips => generated 151 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 28 11:09:05.974989 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:05.975744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:05.979716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:05.980249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:05.987499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:05.987971 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:05.988398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:05.989074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:05.989521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:05.989937 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:06.014025 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lazy-loading 'flavor' on Instance uuid 057e4fde-a7eb-419f-b0ba-ad3e183bf191 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:09:06.198678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f5837366-7041-48eb-ae6f-32c51717eebd tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/action" status: 202 len: 0 microversion: 2.20 time: 0.224221 Oct 28 11:09:06.199294 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 373/735] 10.222.0.46 () {62 vars in 1319 bytes} [Fri Oct 28 11:09:05 2022] POST /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/action => generated 0 bytes in 226 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:09:06.207904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:06.208605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:06.215167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:06.215755 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:06.216408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:06.217105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:06.217643 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:06.218134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:06.349626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:06.357774 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:06.359249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:06.360404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a285d76b-b7c1-4766-9806-fea06758e586 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.156957 Oct 28 11:09:06.361493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 363/736] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:06 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:07.137353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-2075580256", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1437936952", "security_groups": [{"name": "tempest-secgroup-smoke-659594001"}], "networks": [{"uuid": "9ede01c2-68b4-4454-9c54-b9a5972b3253"}], "block_device_mapping_v2": [{"uuid": "887d3cd0-a391-468b-9149-7c98830fdba2", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:07.249096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] validate_networks() for [('9ede01c2-68b4-4454-9c54-b9a5972b3253', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:09:07.386762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:07.387357 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:07.394350 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:07.394824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:07.395251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:07.395933 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:07.396384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:07.396801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:07.524652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:07.533315 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:07.534836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:07.535996 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f313951a-ec8a-4f00-a02d-1ad0fa7be099 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.153188 Oct 28 11:09:07.536563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 364/737] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:07 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:09:07.612215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:09:07.612656 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:09:07.613127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:09:07.613531 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:09:07.613937 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:09:07.622588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:07.623063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:07.623488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:07.624205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:07.624657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:07.625075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:07.641370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Getting quotas for project 9edc5ff928624d87944ee071e578aa17. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:09:07.645251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Getting quotas for user d4ade5932a324a4b8956b5377256ce7a and project 9edc5ff928624d87944ee071e578aa17. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:09:07.702521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:09:07.705586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='887d3cd0-a391-468b-9149-7c98830fdba2',volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:09:07.836026 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0134ba33-ef38-4d0c-bfc9-214914362f96 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.702841 Oct 28 11:09:07.836662 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 374/738] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:09:07 2022] POST /compute/v2.1/servers => generated 405 bytes in 704 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:09:07.848493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ffc2da22-a6d9-4b95-b6ab-4336dd09b22c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:07.849110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ffc2da22-a6d9-4b95-b6ab-4336dd09b22c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:07.902195 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ffc2da22-a6d9-4b95-b6ab-4336dd09b22c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:07.909545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ffc2da22-a6d9-4b95-b6ab-4336dd09b22c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1089 microversion: 2.1 time: 0.065441 Oct 28 11:09:07.910131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 365/739] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:07 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1089 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:08.561589 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:08.562233 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:08.569120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:08.569528 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:08.569963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:08.570673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:08.571125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:08.571551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:08.695500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:08.703540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:08.705061 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:08.706239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6e9228dc-3b8e-4930-b59c-29d11fdca679 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.148575 Oct 28 11:09:08.706808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 375/740] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:08 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:08.933407 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:08.934041 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:08.941791 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:08.942299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:08.942734 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:08.943517 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:08.943919 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:08.944341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:09.010192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:09.018848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b3f4e246-ccf6-462d-857f-6a99a380b383 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1205 microversion: 2.1 time: 0.089531 Oct 28 11:09:09.025612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 366/741] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:08 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1205 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:09.731819 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:09.732417 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:09.739200 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:09.739672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:09.740114 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:09.740798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:09.741248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:09.741667 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:09.945078 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:09.953112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:09.954580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:09.955772 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cd282dd0-72e4-4c93-b563-984a5225b101 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.227852 Oct 28 11:09:09.959932 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 376/742] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:09 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:10.042482 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:10.043102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:10.050082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:10.050563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:10.051205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:10.051897 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:10.052343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:10.052758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:10.184948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:10.193136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a7963e6f-e94e-42fb-b4fa-133e342c253d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.154603 Oct 28 11:09:10.193801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 367/743] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:10 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:10.319843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-102a1e91-fcd9-4fcb-8797-b371d3203b33 req-9ff9f2e1-acfe-4b1d-ba3c-7db942d10da8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c0936af-77e0-4b5c-b84c-08f6f5d72735", "tag": "0404e8da-6840-4714-b253-a87262494023"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:10.330786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-102a1e91-fcd9-4fcb-8797-b371d3203b33 req-9ff9f2e1-acfe-4b1d-ba3c-7db942d10da8 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:10.331263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-102a1e91-fcd9-4fcb-8797-b371d3203b33 req-9ff9f2e1-acfe-4b1d-ba3c-7db942d10da8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:10.331692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-102a1e91-fcd9-4fcb-8797-b371d3203b33 req-9ff9f2e1-acfe-4b1d-ba3c-7db942d10da8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:10.342971 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-102a1e91-fcd9-4fcb-8797-b371d3203b33 req-9ff9f2e1-acfe-4b1d-ba3c-7db942d10da8 service nova] Creating event network-changed:0404e8da-6840-4714-b253-a87262494023 for instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 on hv-cinder-zed-3 Oct 28 11:09:10.348696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-102a1e91-fcd9-4fcb-8797-b371d3203b33 req-9ff9f2e1-acfe-4b1d-ba3c-7db942d10da8 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032648 Oct 28 11:09:10.349315 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 377/744] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:09:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:09:10.981595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:10.982155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:10.988318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:10.988789 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:10.989213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:10.989884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:10.990352 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:10.990767 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:11.115160 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:11.123081 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:11.124503 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:11.125636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-83139b53-d2c9-44a9-8807-9e34e224951c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.148047 Oct 28 11:09:11.126231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 368/745] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:10 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:11.217865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:11.218585 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:11.224690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:11.225189 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:11.225624 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:11.226344 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:11.226823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:11.227243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:11.356994 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:11.364900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-46844ad6-a4bf-428b-94a7-b2c9ad15859a tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.151300 Oct 28 11:09:11.365468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 378/746] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:11 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:12.151154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:12.151792 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:12.158615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:12.159088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:12.159514 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:12.160221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:12.160672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:12.161091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:12.293184 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:12.301955 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:12.303503 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:12.304653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-60839705-a0f5-4b65-a88f-00744e9eac95 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.157447 Oct 28 11:09:12.305209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 369/747] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:12 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:12.387124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:12.387673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:12.394024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:12.394498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:12.394926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:12.395593 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:12.396039 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:12.396455 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:12.524897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:12.532513 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cbec007b-e5b0-419c-9c9d-1f818615fd9c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.149299 Oct 28 11:09:12.534077 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 379/748] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:12 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:13.330492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:13.331094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:13.337489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:13.337961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:13.338417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:13.339092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:13.339535 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:13.339981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:13.464352 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:13.472288 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:13.473709 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:13.474873 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-efb34f5e-4b69-4e79-8cc6-76d5582e64e8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.148821 Oct 28 11:09:13.475429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 370/749] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:13 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:13.556313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:13.556868 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:13.562787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:13.563270 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:13.563701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:13.564891 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:13.565346 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:13.565771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:13.690924 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:13.698368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b081de34-ce52-4d93-b580-50825c990d43 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.146006 Oct 28 11:09:13.698923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 380/750] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:13 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:14.500249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:14.500778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:14.507197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:14.507666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:14.508088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:14.508769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:14.509210 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:14.509623 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:14.639632 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:14.648082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:14.649517 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:14.650701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-36eb818e-346c-458e-9e06-4329880813b0 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.154427 Oct 28 11:09:14.651265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 371/751] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:14 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:14.722053 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:14.722666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:14.728496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:14.728973 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:14.729406 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:14.730112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:14.730561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:14.730977 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:14.860866 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:14.869085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6b5a1e23-f9ea-4459-abf7-62feb936504c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.151342 Oct 28 11:09:14.877090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 381/752] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:14 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:15.676797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:15.677406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:15.683771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:15.684251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:15.684687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:15.685369 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:15.685824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:15.686285 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:15.812951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:15.820995 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:15.822538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:15.823723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9faf4491-5a93-432c-bea3-bfb13200e901 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.151405 Oct 28 11:09:15.824280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 372/753] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:15 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:15.891953 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:15.892540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:15.898789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:15.899261 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:15.899701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:15.900367 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:15.900810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:15.901224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:16.038843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:16.046386 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-af65c9c4-b0f3-4cae-a711-167fb32ebe1c tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.158307 Oct 28 11:09:16.046940 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 382/754] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:15 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:16.849292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:16.849929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:16.856346 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:16.856860 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:16.857302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:16.858009 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:16.858469 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:16.858894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:16.984042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:16.992415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:16.993894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:16.995094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c2b032e2-bf4a-4180-8f96-493c67ecf11e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.150338 Oct 28 11:09:16.995657 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 373/755] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:16 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:17.069219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:17.069874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:17.075799 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:17.076274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:17.076700 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:17.077375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:17.077821 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:17.078272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:17.206308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:17.213750 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-254ce7fe-13c9-4e7a-9ae4-30f26f00589f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.148388 Oct 28 11:09:17.214348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 383/756] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:17 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:18.021952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:18.022589 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:18.025489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:18.034328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:18.034813 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:18.035250 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:18.035947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:18.036401 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:18.036822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:18.169825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:18.177785 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:18.179260 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:18.180406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2df10408-43e5-4044-99f6-5d8afdbd569d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.162356 Oct 28 11:09:18.180957 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 374/757] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:18 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:18.236736 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:18.237620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:18.239588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:18.240242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=6 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:18.246351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:18.246832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:18.247261 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:18.247945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:18.248398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:18.248813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:18.374590 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:18.382821 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0c466b2e-e9fa-406f-a3f2-8e47f7fb0ab4 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.149865 Oct 28 11:09:18.383402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 384/758] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:18 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:19.207932 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:19.208537 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:19.214830 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:19.215299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:19.215726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:19.216399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:19.216844 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:19.217290 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:19.344474 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:19.352949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:19.354402 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:19.355540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2be04d53-3b27-4788-ab61-ff547927746c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.152037 Oct 28 11:09:19.356102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 375/759] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:19 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:19.405618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:19.406296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:19.412715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:19.413186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:19.413615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:19.414322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:19.414773 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:19.415190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:19.540777 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:19.548566 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ca2855d5-6792-4fc5-b144-8fe8054af61e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.146801 Oct 28 11:09:19.549132 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 385/760] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:19 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:20.381123 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:20.381756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:20.388172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:20.388643 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:20.389070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:20.389776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:20.390256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:20.390678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:20.519031 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:20.526857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:20.528280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:20.529415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0b222717-8074-4568-ab2d-8d63a31512d1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.152135 Oct 28 11:09:20.529982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 376/761] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:20 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:20.571025 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:20.571697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:20.577808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:20.578313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:20.578744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:20.580063 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:20.580510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:20.580926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:20.711925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:20.719662 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-aa6b1e95-9d50-4d91-9bc0-5e1aa86ff346 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.152383 Oct 28 11:09:20.720278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 386/762] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:20 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:21.554649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:21.555191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:21.561528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:21.562026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:21.562480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:21.563157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:21.563601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:21.564021 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:21.690681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:21.698609 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:21.700055 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:21.701209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-92926fea-1739-4ab2-a338-490308cec399 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.150527 Oct 28 11:09:21.701774 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 377/763] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:21 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:21.742546 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:21.743181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:21.749030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:21.749502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:21.749928 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:21.750652 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:21.751587 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:21.752006 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:21.879355 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:21.886937 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0ff7fcc3-e6e9-4b29-8ebc-47e8aae35762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.148267 Oct 28 11:09:21.887502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 387/764] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:21 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:22.234796 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-17edf9a0-17e9-44f4-8c42-c4efbf96311d req-86e7a6ef-e437-4a4a-976e-cd22ab46cf65 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c0936af-77e0-4b5c-b84c-08f6f5d72735", "name": "network-vif-plugged", "status": "completed", "tag": "0404e8da-6840-4714-b253-a87262494023"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:22.245653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-17edf9a0-17e9-44f4-8c42-c4efbf96311d req-86e7a6ef-e437-4a4a-976e-cd22ab46cf65 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:22.246472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-17edf9a0-17e9-44f4-8c42-c4efbf96311d req-86e7a6ef-e437-4a4a-976e-cd22ab46cf65 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:22.246903 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-17edf9a0-17e9-44f4-8c42-c4efbf96311d req-86e7a6ef-e437-4a4a-976e-cd22ab46cf65 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:22.258290 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-17edf9a0-17e9-44f4-8c42-c4efbf96311d req-86e7a6ef-e437-4a4a-976e-cd22ab46cf65 service nova] Creating event network-vif-plugged:0404e8da-6840-4714-b253-a87262494023 for instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 on hv-cinder-zed-3 Oct 28 11:09:22.263620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-17edf9a0-17e9-44f4-8c42-c4efbf96311d req-86e7a6ef-e437-4a4a-976e-cd22ab46cf65 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032451 Oct 28 11:09:22.264304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 378/765] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:09:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:09:22.726953 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:22.727484 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:22.733290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:22.733759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:22.734281 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:22.734950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:22.735401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:22.735919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:22.859364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:22.863566 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:22.868860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:22.870328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:22.871472 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-71a89069-da91-47ad-a5a2-5fc46209444e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1742 microversion: 2.20 time: 0.148428 Oct 28 11:09:22.872033 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 388/766] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:22 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1742 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:22.910630 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:22.911326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:22.917823 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:22.918336 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:22.918767 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:22.919448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:22.919899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:22.920318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:23.051501 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:23.059192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3190078e-8b62-4653-9284-728487084a89 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.152940 Oct 28 11:09:23.059754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 379/767] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:22 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:23.897954 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:23.898570 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:23.904831 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:23.905308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:23.905738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:23.906432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:23.906879 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:23.907295 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:24.031538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:24.040016 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:24.041464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:24.042651 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7d44802d-c3ab-4572-81bf-23b35543b31c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1742 microversion: 2.20 time: 0.149220 Oct 28 11:09:24.043215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 389/768] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:23 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1742 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:24.083678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:24.084625 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:24.090820 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:24.091293 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:24.091720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:24.092405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:24.092853 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:24.093268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:24.111834 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:24.224436 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:24.232188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-bf829037-b298-4c27-a172-39d7b3248ad7 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1260 microversion: 2.1 time: 0.152328 Oct 28 11:09:24.232762 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 380/769] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:24 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:25.068550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:25.069163 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:25.075547 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:25.076034 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:25.076471 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:25.077160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:25.077613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:25.078072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:25.206548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:25.214580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:25.216013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:25.217195 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9e27981c-75c6-4bfd-889e-71e1167328af tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1742 microversion: 2.20 time: 0.152652 Oct 28 11:09:25.217747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 390/770] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:25 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1742 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:25.255656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:25.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:25.262725 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:25.263204 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:25.263633 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:25.264319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:25.264770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:25.265192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:25.392201 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:25.399828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5902c847-f0c5-4b32-acbc-5c65cad41297 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1443 microversion: 2.1 time: 0.148108 Oct 28 11:09:25.400388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 381/771] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:25 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1443 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:25.422279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:25.422885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:25.428653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:25.429125 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:25.429550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:25.430254 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:25.430703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:25.431116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:25.557246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:25.564838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ed333a35-ba15-476b-9728-a8018e47acce tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1443 microversion: 2.1 time: 0.146348 Oct 28 11:09:25.565399 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 391/772] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:25 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1443 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:26.243394 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:26.244231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:26.250490 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:26.252272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:26.252272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:26.252825 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:26.253371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:26.253799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:26.271731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-35b2c40c-bb77-47cf-b12a-9f1798c6cb9b req-9baccb46-8ff9-4151-8cd4-6b5e7ce388cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "057e4fde-a7eb-419f-b0ba-ad3e183bf191", "name": "network-vif-plugged", "status": "completed", "tag": "ded90a4e-c044-4824-8ed3-a7277034c619"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:26.282832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-35b2c40c-bb77-47cf-b12a-9f1798c6cb9b req-9baccb46-8ff9-4151-8cd4-6b5e7ce388cd service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:26.283688 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-35b2c40c-bb77-47cf-b12a-9f1798c6cb9b req-9baccb46-8ff9-4151-8cd4-6b5e7ce388cd service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:26.284665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-35b2c40c-bb77-47cf-b12a-9f1798c6cb9b req-9baccb46-8ff9-4151-8cd4-6b5e7ce388cd service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:26.304037 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-35b2c40c-bb77-47cf-b12a-9f1798c6cb9b req-9baccb46-8ff9-4151-8cd4-6b5e7ce388cd service nova] Creating event network-vif-plugged:ded90a4e-c044-4824-8ed3-a7277034c619 for instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 on hv-cinder-zed-3 Oct 28 11:09:26.310521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-35b2c40c-bb77-47cf-b12a-9f1798c6cb9b req-9baccb46-8ff9-4151-8cd4-6b5e7ce388cd service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043385 Oct 28 11:09:26.311359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 392/773] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:09:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:09:26.402511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:26.410871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:26.412330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:26.413475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-73b7b9d5-34c2-4a87-a9e6-282586fcc4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1742 microversion: 2.20 time: 0.174346 Oct 28 11:09:26.414049 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 382/774] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:26 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1742 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:27.439990 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:27.440795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:27.446832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:27.447316 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:27.447753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:27.448447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:27.448902 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:27.449328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:27.546188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-3ce8c3f5-5a02-4d64-8070-fdfaf4d7f459 req-a7f591e2-5f69-46d1-a3aa-e010dc241bb3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c0936af-77e0-4b5c-b84c-08f6f5d72735", "tag": "0404e8da-6840-4714-b253-a87262494023"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:27.557301 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-3ce8c3f5-5a02-4d64-8070-fdfaf4d7f459 req-a7f591e2-5f69-46d1-a3aa-e010dc241bb3 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:27.557845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-3ce8c3f5-5a02-4d64-8070-fdfaf4d7f459 req-a7f591e2-5f69-46d1-a3aa-e010dc241bb3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:27.558368 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-3ce8c3f5-5a02-4d64-8070-fdfaf4d7f459 req-a7f591e2-5f69-46d1-a3aa-e010dc241bb3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:27.569798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-3ce8c3f5-5a02-4d64-8070-fdfaf4d7f459 req-a7f591e2-5f69-46d1-a3aa-e010dc241bb3 service nova] Creating event network-changed:0404e8da-6840-4714-b253-a87262494023 for instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 on hv-cinder-zed-3 Oct 28 11:09:27.574886 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-3ce8c3f5-5a02-4d64-8070-fdfaf4d7f459 req-a7f591e2-5f69-46d1-a3aa-e010dc241bb3 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032355 Oct 28 11:09:27.575810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 383/775] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:09:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:09:27.642299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:27.650389 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:27.651831 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:27.652983 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-745181f0-c7f4-4383-8fab-db891f0456de tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1742 microversion: 2.20 time: 0.217426 Oct 28 11:09:27.653549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 393/776] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:27 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1742 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:28.682077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:28.682778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:28.688900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:28.689379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:28.689830 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:28.690532 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:28.690986 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:28.691409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:28.823256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:28.831761 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:28.833182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:28.834354 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f101578b-9991-4e07-9f09-e1187576247b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1742 microversion: 2.20 time: 0.157173 Oct 28 11:09:28.834910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 384/777] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:28 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1742 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:29.860116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:29.861572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:29.868263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:29.869041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:29.869673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:29.870614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:29.871324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:29.872094 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:30.105586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:30.118324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:30.120274 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:30.121743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fe473cf3-dd2e-400c-8eac-d102b104e621 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.265509 Oct 28 11:09:30.122706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 394/778] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:29 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:09:31.149220 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:31.149739 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:31.156730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:31.157775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:31.158874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:31.160182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:31.160655 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:31.161086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:31.306248 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:31.315092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:31.316913 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:31.318282 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5681597a-5127-43fa-9bb1-035789458740 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.173679 Oct 28 11:09:31.319304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 385/779] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:31 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:32.345121 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:32.345717 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:32.352194 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:32.352664 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:32.353095 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:32.353815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:32.354298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:32.354724 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:32.515718 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:32.527730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:32.529380 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:32.532157 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9e0d222c-07c3-435e-89c3-c692b694bb24 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.190240 Oct 28 11:09:32.546508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 395/780] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:32 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:33.562859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:33.564392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:33.568375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=51 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:33.578681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:33.579912 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:33.580876 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:33.582206 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:33.582847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:33.583425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:33.757562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:33.767904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:33.769343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:33.779384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ebce30af-0046-41b0-8d2f-2152c4f9f500 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.220457 Oct 28 11:09:33.780046 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 386/781] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:33 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:34.805629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:34.806186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:34.812357 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:34.813058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:34.813521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:34.814413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:34.814864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:34.815485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:34.961615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:34.971871 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:34.973482 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:34.974777 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-38356dbb-3215-487b-baa7-3da723eb8f9d tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.173193 Oct 28 11:09:34.976737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 396/782] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:34 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:36.021668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:36.022379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:36.029061 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:36.029642 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:36.030098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:36.030793 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:36.031236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:36.031753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:36.175983 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:36.185037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:36.186942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:36.188116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-167ad5e5-25e0-4cd4-b912-e361241c461e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.170363 Oct 28 11:09:36.202054 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 387/783] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:36 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:37.215572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:37.216975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:37.219006 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:37.226966 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:37.228473 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:37.229343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:37.231777 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:37.232638 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:37.233172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:37.391477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:37.400295 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:37.401834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:37.403093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-528f546c-2c55-413d-92b3-5bfac62e8aa3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.191498 Oct 28 11:09:37.403885 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 397/784] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:37 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:38.436118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:38.437728 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:38.448278 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:38.449241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:38.450458 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:38.455568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:38.456676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:38.457335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:38.656109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:38.666496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:38.668076 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:38.669234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2236d0a9-4165-47bc-8d4d-366c97d18f56 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.238274 Oct 28 11:09:38.670224 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 388/785] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:38 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:09:39.695635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:39.696897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:39.703136 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:39.703875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:39.704306 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:39.704982 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:39.705427 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:39.705843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:39.853456 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:39.862970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:39.864698 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:39.866064 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-287e2949-ec96-4556-b844-33b61314ab54 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.174338 Oct 28 11:09:39.866837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 398/786] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:39 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:40.890658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:40.891780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:40.898075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:40.898556 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:40.898989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:40.899670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:40.900214 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:40.900628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:41.024462 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:41.033027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:41.034492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:41.035641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-26d3541a-2ef0-42e0-894f-026b886b34cb tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1737 microversion: 2.20 time: 0.148954 Oct 28 11:09:41.036200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 389/787] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:40 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1737 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:09:42.060648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:42.061285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:42.067797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:42.068272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:42.068701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:42.069383 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:42.069835 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:42.070280 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:42.207827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:42.216064 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:42.217498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:42.218669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-441a9c91-17dd-43cd-bac6-f1e503e0f9ef tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1719 microversion: 2.20 time: 0.161945 Oct 28 11:09:42.219238 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 399/788] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:42 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1719 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:42.850869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-157278796", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1118103513", "security_groups": [{"name": "tempest-secgroup-smoke-1525806302"}], "block_device_mapping_v2": [{"uuid": "f8343644-4b92-4705-ab21-b63cec182d24", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "447ffcd3-d772-4c15-92cb-558dd17897d4"}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:42.960311 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] validate_networks() for [('447ffcd3-d772-4c15-92cb-558dd17897d4', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:09:43.243901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:43.244503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:43.250640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:43.251123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:43.251558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:43.252245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:43.252689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:43.253106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:43.331864 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:09:43.332276 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:09:43.332740 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:09:43.333140 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:09:43.333544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:09:43.342620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:43.343099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:43.343607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:43.344314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:43.345406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:43.345828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:43.364159 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:09:43.368335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Getting quotas for user 6b926f7161c0487a8c2b04ada3b374e3 and project 9aff70e0d61c41fbb469ed4bcd785b23. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:09:43.394662 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:43.404689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:43.406295 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:43.408663 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cd506efa-283a-46e6-8276-2db2aeb67d35 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1719 microversion: 2.20 time: 0.167327 Oct 28 11:09:43.409244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 400/789] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:43 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1719 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:09:43.428286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:09:43.430450 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='f8343644-4b92-4705-ab21-b63cec182d24',volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:09:43.558921 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR oslo.messaging._drivers.impl_rabbit [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [84064682-282a-422c-a3dc-c61eacf244e3] AMQP server on 10.222.0.46:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 28 11:09:44.274653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-e08deb7c-cb7c-49de-b466-01095ee3519b req-46d1881d-730e-42c3-aa3a-84f38cab6b8f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "057e4fde-a7eb-419f-b0ba-ad3e183bf191", "name": "network-vif-unplugged", "status": "completed", "tag": "ded90a4e-c044-4824-8ed3-a7277034c619"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:44.284244 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e08deb7c-cb7c-49de-b466-01095ee3519b req-46d1881d-730e-42c3-aa3a-84f38cab6b8f service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:44.284710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e08deb7c-cb7c-49de-b466-01095ee3519b req-46d1881d-730e-42c3-aa3a-84f38cab6b8f service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:44.285156 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e08deb7c-cb7c-49de-b466-01095ee3519b req-46d1881d-730e-42c3-aa3a-84f38cab6b8f service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:44.296090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.compute.server_external_events [req-e08deb7c-cb7c-49de-b466-01095ee3519b req-46d1881d-730e-42c3-aa3a-84f38cab6b8f service nova] Unable to find a host for instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191. Dropping event network-vif-unplugged {{(pid=120543) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 28 11:09:44.296966 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-e08deb7c-cb7c-49de-b466-01095ee3519b req-46d1881d-730e-42c3-aa3a-84f38cab6b8f service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.026101 Oct 28 11:09:44.297552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 401/790] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:09:44 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 27 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Oct 28 11:09:44.433682 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:44.434276 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:44.440651 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:44.441131 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:44.441569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:44.442277 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:44.442731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:44.443154 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:44.567617 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO oslo.messaging._drivers.impl_rabbit [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [84064682-282a-422c-a3dc-c61eacf244e3] Reconnected to AMQP server on 10.222.0.46:5672 via [amqp] client with port 34390. Oct 28 11:09:44.568605 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:44.572210 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-be89472d-88b2-4aac-a45e-1507223236e2 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.725163 Oct 28 11:09:44.572867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 390/791] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:09:42 2022] POST /compute/v2.1/servers => generated 406 bytes in 1726 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:09:44.577450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:44.578978 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:44.580156 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2099104e-8499-4ba8-bd80-82a5ef64b9a2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1662 microversion: 2.20 time: 0.150392 Oct 28 11:09:44.580867 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 402/792] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:44 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1662 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:44.583924 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c3b60571-37ae-4bb9-9403-0f347fb0626c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:44.584538 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c3b60571-37ae-4bb9-9403-0f347fb0626c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:44.606006 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "61e9be6d-ab98-4e4b-b806-9f58521de8b0"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:44.608304 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:44.615070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:44.615550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:44.615980 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:44.616658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:44.617300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:44.618074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:44.640103 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c3b60571-37ae-4bb9-9403-0f347fb0626c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:44.647207 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c3b60571-37ae-4bb9-9403-0f347fb0626c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1094 microversion: 2.1 time: 0.066760 Oct 28 11:09:44.647860 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 391/793] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:44 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1094 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:09:44.920171 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a0f20186-1267-4578-a4ef-9218340c4742 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.318133 Oct 28 11:09:44.920871 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 403/794] 10.222.0.46 () {62 vars in 1364 bytes} [Fri Oct 28 11:09:44 2022] POST /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-volume_attachments => generated 188 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 28 11:09:45.052339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:45.052837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:45.059755 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.060237 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.060669 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.061376 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.061812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.062277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.338088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6ef606ac-47fa-4fea-b13d-b347d30f69f4 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-volume_attachments/61e9be6d-ab98-4e4b-b806-9f58521de8b0" status: 202 len: 0 microversion: 2.20 time: 0.290254 Oct 28 11:09:45.338702 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 392/795] 10.222.0.46 () {60 vars in 1457 bytes} [Fri Oct 28 11:09:45 2022] DELETE /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-volume_attachments/61e9be6d-ab98-4e4b-b806-9f58521de8b0 => generated 0 bytes in 292 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:09:45.399226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:45.399817 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:45.405648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.406175 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.406609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.407934 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.408381 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.408802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.412319 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:45.439556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120543) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 28 11:09:45.504027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-da81f461-bcf9-4378-9008-d4fc7fa82f3a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "POST /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/action" status: 202 len: 0 microversion: 2.20 time: 0.108555 Oct 28 11:09:45.504553 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 404/796] 10.222.0.46 () {62 vars in 1319 bytes} [Fri Oct 28 11:09:45 2022] POST /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/action => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:09:45.512794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:45.513405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:45.519379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.519864 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.520299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.520992 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.521475 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.521859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.660936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:45.669947 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:45.670607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:45.677480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:45.677666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.678236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.678843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.679315 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:45.679527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:45.679977 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:45.680400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:45.680777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-17563260-3d08-489d-8e08-2ebba6ba42c9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1670 microversion: 2.20 time: 0.171909 Oct 28 11:09:45.681561 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 393/797] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:45 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1670 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:45.802521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:45.811302 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b223de2a-a61c-4228-860b-85d4c3d0c438 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1200 microversion: 2.1 time: 0.145903 Oct 28 11:09:45.812469 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 405/798] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:45 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1200 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:09:46.707794 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:46.708517 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:46.715212 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:46.715770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:46.716208 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:46.716926 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:46.717374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:46.717799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:46.835509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:46.836291 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:46.842340 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:46.843118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:46.843615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:46.844454 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:46.845680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:46.846197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:46.854748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:46.863366 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:46.864957 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:46.866237 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fb62c89b-bab9-4a90-9429-a4c1b01aee9f tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.162565 Oct 28 11:09:46.866803 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 394/799] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:46 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:47.058411 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:47.068102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9232f324-1e91-46bf-bbc4-158a66051d7f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.236372 Oct 28 11:09:47.069005 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 406/800] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:46 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:47.374417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-eaf867ad-fcb9-4a90-a6fe-08c4d5b4d7eb req-5af5f10b-365e-4fb0-9d96-8eeecaf2988d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5fb835d7-4e01-4735-9093-682fe2f848e0", "tag": "fefb02ab-67c7-4cc8-91df-d3860ccf795b"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:09:47.389113 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-eaf867ad-fcb9-4a90-a6fe-08c4d5b4d7eb req-5af5f10b-365e-4fb0-9d96-8eeecaf2988d service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:47.389771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-eaf867ad-fcb9-4a90-a6fe-08c4d5b4d7eb req-5af5f10b-365e-4fb0-9d96-8eeecaf2988d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:47.390411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-eaf867ad-fcb9-4a90-a6fe-08c4d5b4d7eb req-5af5f10b-365e-4fb0-9d96-8eeecaf2988d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:47.402431 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-eaf867ad-fcb9-4a90-a6fe-08c4d5b4d7eb req-5af5f10b-365e-4fb0-9d96-8eeecaf2988d service nova] Creating event network-changed:fefb02ab-67c7-4cc8-91df-d3860ccf795b for instance 5fb835d7-4e01-4735-9093-682fe2f848e0 on hv-cinder-zed-3 Oct 28 11:09:47.409206 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-eaf867ad-fcb9-4a90-a6fe-08c4d5b4d7eb req-5af5f10b-365e-4fb0-9d96-8eeecaf2988d service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038886 Oct 28 11:09:47.410185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 395/801] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:09:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:09:47.891914 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:47.893090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:47.904632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:47.907625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:47.908246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:47.915172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:47.915876 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:47.916488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:48.075679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:48.088429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:48.090480 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:48.091834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b664a941-8141-46e6-85f9-7ff2ebaabd69 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.203920 Oct 28 11:09:48.092571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 407/802] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:47 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:48.106230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:48.106230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:48.111119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:48.111819 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:48.117536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:48.118327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:48.118795 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:48.119226 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:48.286277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:48.294235 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b7cda4c1-1f44-4748-81c2-24ebdf11588d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.204839 Oct 28 11:09:48.294935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 396/803] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:48 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:09:49.120247 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:49.120831 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:49.126750 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:49.127222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:49.127646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:49.128314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:49.128755 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:49.129169 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:49.259864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:49.268532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:49.269998 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:49.271173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0598789c-49b1-4e66-9b82-75197f501983 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.154789 Oct 28 11:09:49.271913 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 408/804] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:49 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:49.320086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:49.320836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:49.327503 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:49.328237 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:49.328702 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:49.329402 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:49.329851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:49.330303 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:49.464427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:49.466768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:49.474327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3eb03fca-c45c-4735-885e-feb702547e43 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.158094 Oct 28 11:09:49.475453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 397/805] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:49 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:50.296254 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:50.296843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:50.303362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:50.303833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:50.304260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:50.304936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:50.305377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:50.305789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:50.437027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:50.446145 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:50.447633 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:50.448800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c9a84f1c-9bc5-4af7-892b-d1e6e4656dac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.156410 Oct 28 11:09:50.449359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 409/806] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:50 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:50.497895 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:50.498530 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:50.506863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:50.507548 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:50.507976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:50.508662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:50.509108 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:50.509520 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:50.651154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:50.658984 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-943db516-4c73-47cf-952a-ea910565b010 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.164845 Oct 28 11:09:50.659550 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 398/807] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:50 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:51.475420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:51.477090 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:51.483691 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:51.484246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:51.484748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:51.485523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:51.486067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:51.486551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:51.618675 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:51.627122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:51.628574 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:51.629773 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-42482835-4d1b-4fc6-8bc7-2ebc0fc11dc7 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.159409 Oct 28 11:09:51.630452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 410/808] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:51 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:51.683650 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:51.684267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:51.692092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:51.692575 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:51.693013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:51.693713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:51.694208 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:51.694635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:51.826732 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:51.836521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e7aa2b4b-c37b-4d53-bca1-cc8471618d34 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.157508 Oct 28 11:09:51.837268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 399/809] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:51 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:52.655736 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:52.657079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:52.663360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:52.663923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:52.664417 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:52.665173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:52.665644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:52.666442 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:52.798169 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:52.806633 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:52.808102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:52.809249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a8d63a9d-1d87-4741-a870-626faa31e6c1 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.157521 Oct 28 11:09:52.809815 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 411/810] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:52 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 28 11:09:52.859439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:52.860158 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:52.867888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:52.868378 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:52.868816 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:52.869519 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:52.869988 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:52.870421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:53.003103 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:53.011037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9b2b8c89-1edd-4422-836b-0a45a30b5c6a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.155525 Oct 28 11:09:53.011620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 400/811] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:52 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:53.838342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:53.839073 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:53.845610 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:53.846122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:53.847384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:53.848152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:53.848609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:53.849033 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:53.983759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:53.992900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:53.994413 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:53.995585 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-82b83dac-9d44-49ee-ac57-232be6c4e24e tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.162824 Oct 28 11:09:53.996150 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 412/812] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:53 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:54.033605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:54.034415 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:54.036182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:54.042531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:54.043168 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:54.044703 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:54.045633 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:54.046147 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:54.046788 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:54.183389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:54.191924 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5b7e704d-15e3-4933-ae3e-0a76027ed046 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.162095 Oct 28 11:09:54.192497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 401/813] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:54 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:55.021522 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:55.022296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:55.028805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:55.029308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:55.029810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:55.030623 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:55.031083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:55.031516 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:55.173523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:55.181950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:55.183813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:55.185310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-11d7242c-8307-456b-aa00-0c257bb80dc9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.168356 Oct 28 11:09:55.186001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 413/814] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:55 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:55.216261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:55.216823 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:55.218634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=8 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:09:55.224914 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:55.225399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:55.225832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:55.226550 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:55.226998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:55.227417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:55.373000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:55.382960 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-598da559-0095-4dec-b96d-49cf0a0a1af9 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.170528 Oct 28 11:09:55.383546 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 402/815] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:55 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:56.211288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:56.212022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:56.218138 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:56.218626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:56.219064 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:56.219762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:56.220217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:56.220644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:56.357434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:56.366350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:56.367805 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:56.369140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dac6dbde-cb36-4c46-90a2-91a7ce545c4c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.162410 Oct 28 11:09:56.369707 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 414/816] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:56 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:56.406503 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:56.407151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:56.413785 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:56.414296 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:56.414856 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:56.415738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:56.416192 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:56.416610 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:56.558938 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:56.567067 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-83364762-29d2-4adc-aecb-68f3f66a4ab7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.163834 Oct 28 11:09:56.567636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 403/817] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:56 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:57.394600 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:57.395203 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:57.402133 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:57.402612 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:57.403044 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:57.403731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:57.404191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:57.404615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:57.537796 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:57.545908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:57.547284 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:57.548439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9e33a5a9-1364-4832-a935-50a478611ef8 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.157860 Oct 28 11:09:57.549177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 415/818] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:57 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:57.589198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:57.589817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:57.596193 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:57.596670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:57.597102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:57.597785 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:57.598267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:57.598896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:57.739265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:57.747190 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-01e83e96-72c5-4014-82d2-4b5edf17a26f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.161781 Oct 28 11:09:57.747763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 404/819] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:57 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:58.574385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:58.576166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:58.584258 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:58.585324 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:58.586067 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:58.587022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:58.587695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:58.588334 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:58.756669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:58.769500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:58.770305 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:58.777904 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:58.778269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:58.778429 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:58.778857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:58.779548 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:58.779738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:58.779999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:58.780442 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:58.780988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cf6c90a9-cf69-4f06-87d2-0726f6023cf6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.211564 Oct 28 11:09:58.781548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 416/820] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:58 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:09:58.912402 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:58.919967 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-476e3398-9a02-4441-9490-05377118b9a8 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.154270 Oct 28 11:09:58.920528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 405/821] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:58 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:09:59.807109 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:59.807823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:59.813765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:59.814272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:59.814800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:59.815490 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:59.815941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:59.816363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:59.943645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:09:59.944624 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:09:59.952179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:59.953097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:59.953366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:59.953845 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:59.954631 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:09:59.955114 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:09:59.955552 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:09:59.964738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:59.966863 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:09:59.968411 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-49edc0f8-4ab2-48ac-b37f-4bb159c6a3ac tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.165205 Oct 28 11:09:59.982001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 417/822] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:09:59 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:00.094011 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:00.101727 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f16177ce-261d-4edc-9434-cfbfeb1a567e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.162469 Oct 28 11:10:00.102434 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 406/823] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:09:59 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:00.994551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:00.995161 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:01.001013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:01.001497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:01.001929 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:01.002635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:01.003088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:01.003509 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:01.124974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:01.125719 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:01.131686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:01.132330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:01.132812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:01.133244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:01.133944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:01.134438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:01.134849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:01.140118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:01.141557 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:01.142730 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b769b77a-cae2-4c78-a206-d825ad665825 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.152058 Oct 28 11:10:01.143293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 418/824] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:00 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:01.260710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:01.268412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-497ed07b-5010-4d77-a06f-4a4030d47a31 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.147481 Oct 28 11:10:01.268970 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 407/825] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:01 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:02.167672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:02.168285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:02.174204 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:02.174682 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:02.175118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:02.175833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:02.176286 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:02.176853 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:02.291164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:02.291765 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:02.298299 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:02.299010 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:02.299509 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:02.300268 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:02.300796 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:02.301314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:02.314032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:02.318220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:02.323861 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:02.325345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:02.326542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d0809e4c-4cbd-46dc-96c5-a2f2999509da tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.162860 Oct 28 11:10:02.327109 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 419/826] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:02 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:02.431294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:02.438888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2df9ea5b-a4d3-484b-886e-3ed0704fcdad tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.151825 Oct 28 11:10:02.439446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 408/827] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:02 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:03.354398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:03.355013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:03.361582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:03.362084 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:03.362772 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:03.363461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:03.363916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:03.364332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:03.461769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:03.462395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:03.468676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:03.469150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:03.469579 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:03.470290 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:03.470736 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:03.471154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:03.491268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:03.499315 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:03.500773 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:03.501938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cf8125da-ce01-4006-8677-e0f84db109f9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.152658 Oct 28 11:10:03.502547 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 420/828] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:03 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:03.598302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:03.605991 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b3ee8781-ce62-4a80-8483-27e7559b0261 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.148491 Oct 28 11:10:03.606565 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 409/829] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:03 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:04.527080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:04.527661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:04.533457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:04.533932 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:04.534398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:04.535074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:04.535524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:04.536497 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:04.629894 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:04.630661 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:04.633209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:04.638314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:04.638790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:04.639216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:04.639899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:04.640345 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:04.640758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:04.663740 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:04.671828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:04.673268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:04.674447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fad3951b-7c7b-4e4d-ab8f-d144dae6c5e6 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.151359 Oct 28 11:10:04.675009 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 421/830] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:04 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:04.769663 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:04.777947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-eef9ac7d-d927-452c-98dc-7431b51f0da3 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.151812 Oct 28 11:10:04.778486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 410/831] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:04 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:05.699561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:05.700167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:05.706280 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:05.706772 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:05.707210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:05.708446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:05.708899 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:05.709356 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:05.800579 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:05.801190 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:05.807817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:05.808298 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:05.808728 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:05.809421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:05.809871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:05.810328 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:05.838970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:05.847251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:05.848812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:05.850001 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b49883fc-7a04-4d02-aa14-e49891e83a25 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.154322 Oct 28 11:10:05.850562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 422/832] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:05 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:05.938989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:05.947108 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-924d3704-e96c-4fc4-9fe3-ed155afb91c4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.150352 Oct 28 11:10:05.947699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 411/833] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:05 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:06.875359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:06.876034 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:06.882411 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:06.882876 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:06.883300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:06.883970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:06.884414 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:06.884828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:06.970862 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:06.971497 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:06.978130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:06.978611 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:06.979045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:06.979727 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:06.980172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:06.980590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:07.011252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:07.019308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:07.020751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:07.021912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-753299b8-5619-48db-8bbc-9a641e9fdd75 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.150515 Oct 28 11:10:07.022508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 423/834] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:06 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:07.108199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:07.116531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b930c3a8-967d-482e-bed0-9b97ed65517e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.149587 Oct 28 11:10:07.117099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 412/835] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:06 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:08.047823 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:08.048409 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:08.050270 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:08.055807 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:08.056283 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:08.056717 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:08.057956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:08.058434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:08.058858 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:08.139855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:08.140459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:08.146821 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:08.147298 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:08.147726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:08.148410 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:08.148863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:08.149277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:08.191752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:08.199944 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:08.201398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:08.202598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ef3382bd-4f05-4c4c-8919-e605e7218119 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.158746 Oct 28 11:10:08.203164 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 424/836] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:08 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:08.281924 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:08.290201 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3dc1d872-3237-43e4-9e81-d43e9c611d0e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.154205 Oct 28 11:10:08.290776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 413/837] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:08 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:09.228156 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:09.228760 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:09.234542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:09.235014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:09.235443 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:09.236119 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:09.236561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:09.236976 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:09.312728 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:09.313444 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:09.319872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:09.320358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:09.320793 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:09.321487 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:09.321942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:09.322391 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:09.337317 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=100 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:09.363705 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:09.371873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:09.373323 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:09.374534 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-bafd14ad-86ba-48f7-8c2c-3f27683300ff tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.150225 Oct 28 11:10:09.375099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 425/838] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:09 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:09.450292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:09.458363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-baf776f5-c5e8-4574-b35d-873ad5e24b89 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.149004 Oct 28 11:10:09.458921 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 414/839] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:09 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:10.399532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:10.400135 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:10.406091 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:10.406567 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:10.406994 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:10.407786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:10.408235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:10.408650 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:10.480802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:10.481717 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:10.489674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:10.490392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:10.490901 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:10.491658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:10.492177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:10.492665 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:10.542076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:10.550595 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:10.552699 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:10.554104 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b8481260-7644-4fa9-8dc5-33bbbe44ba9a tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.158468 Oct 28 11:10:10.554667 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 426/840] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:10 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:10.559424 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-1283c077-8bde-4325-a8f2-24ecdf05aaab req-85efcc4a-ab57-46e6-8ae6-5e3098d80abe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "057e4fde-a7eb-419f-b0ba-ad3e183bf191", "name": "network-vif-plugged", "status": "completed", "tag": "ded90a4e-c044-4824-8ed3-a7277034c619"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:10:10.568905 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-1283c077-8bde-4325-a8f2-24ecdf05aaab req-85efcc4a-ab57-46e6-8ae6-5e3098d80abe service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:10.569614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-1283c077-8bde-4325-a8f2-24ecdf05aaab req-85efcc4a-ab57-46e6-8ae6-5e3098d80abe service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:10.570226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-1283c077-8bde-4325-a8f2-24ecdf05aaab req-85efcc4a-ab57-46e6-8ae6-5e3098d80abe service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:10.589398 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-1283c077-8bde-4325-a8f2-24ecdf05aaab req-85efcc4a-ab57-46e6-8ae6-5e3098d80abe service nova] Creating event network-vif-plugged:ded90a4e-c044-4824-8ed3-a7277034c619 for instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 on hv-cinder-zed-3 Oct 28 11:10:10.594746 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-1283c077-8bde-4325-a8f2-24ecdf05aaab req-85efcc4a-ab57-46e6-8ae6-5e3098d80abe service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038744 Oct 28 11:10:10.595350 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 427/841] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:10:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:10:10.632580 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:10.640165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-9b5aca8b-1054-4c98-883e-132bb20b1a4a tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.163332 Oct 28 11:10:10.640737 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 415/842] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:10 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:11.579712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:11.580404 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:11.586543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:11.587043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:11.587475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:11.588158 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:11.588605 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:11.589024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:11.662899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:11.663495 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:11.670001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:11.670486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:11.670920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:11.671608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:11.672062 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:11.672483 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:11.715900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:11.723901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:11.725333 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:11.726584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8adb79a1-2479-4bbd-82ff-eaebeb47e7f2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.151176 Oct 28 11:10:11.727232 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 428/843] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:11 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:11.801700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:11.809295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f8f75e7b-18cc-4936-a439-6683aea5782c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.150475 Oct 28 11:10:11.809866 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 416/844] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:11 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:12.752524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:12.753126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:12.759651 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:12.760122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:12.760547 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:12.761222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:12.761668 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:12.762122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:12.833218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:12.833857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:12.840465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:12.840944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:12.841371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:12.842210 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:12.842660 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:12.843083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:12.894194 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:12.903248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:12.904690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:12.905846 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6576059b-0092-4c51-944b-4632ecb0619c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.157189 Oct 28 11:10:12.906673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 429/845] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:12 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:12.975120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:12.983218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a9d56444-0106-44a4-9100-becff3859969 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.154392 Oct 28 11:10:12.983876 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 417/846] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:12 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:13.931727 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:13.932406 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:13.938789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:13.939255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:13.939682 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:13.940363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:13.940806 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:13.941265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:14.006708 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:14.007313 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:14.014716 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:14.015316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:14.015878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:14.016695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:14.017267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:14.017692 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:14.074317 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:14.083320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:14.084819 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:14.086082 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9bae703f-5513-4d99-9faa-149b7c1549c2 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1728 microversion: 2.20 time: 0.158548 Oct 28 11:10:14.087004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 430/847] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:13 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1728 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:14.151641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:14.159284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e442b409-4c42-4e2c-8794-e3bf63397671 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.157019 Oct 28 11:10:14.159841 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 418/848] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:14 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:14.598960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:14.599450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:14.605456 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:14.605936 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:14.606401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:14.607094 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:14.607549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:14.607968 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:14.630810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-63e65b92-b395-490d-ba58-2e3ce35a4622 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/ips" status: 200 len: 144 microversion: 2.1 time: 0.036328 Oct 28 11:10:14.631412 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 431/849] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:10:14 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/ips => generated 144 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:10:15.111782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:15.112498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:15.119222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:15.119695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:15.120123 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:15.120797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:15.121241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:15.121651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:15.182437 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:15.182993 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:15.188915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:15.189391 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:15.189824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:15.190533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:15.190984 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:15.191405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:15.250961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:15.260281 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:15.261738 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:15.262910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-92b6f9cd-7f30-46d0-b5ca-6f3b1519727c tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1715 microversion: 2.20 time: 0.154996 Oct 28 11:10:15.263469 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 419/850] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:10:15 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1715 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:10:15.324992 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:15.332647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1efb3462-c5ad-4eac-997f-6fa2b03cd450 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.154154 Oct 28 11:10:15.333206 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 432/851] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:15 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:16.355142 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:16.355761 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:16.365962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:16.366483 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:16.366910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:16.367591 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:16.368036 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:16.368451 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:16.505495 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:16.513953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d6b90ab5-19c3-4e8a-967f-f4c2e8b86338 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.162571 Oct 28 11:10:16.514547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 420/852] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:16 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:16.560847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-ee7e9605-acdf-435e-9196-297c1c0f2917 req-5d055830-5a6b-4277-86af-a878aed609b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5fb835d7-4e01-4735-9093-682fe2f848e0", "name": "network-vif-plugged", "status": "completed", "tag": "fefb02ab-67c7-4cc8-91df-d3860ccf795b"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:10:16.570288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-ee7e9605-acdf-435e-9196-297c1c0f2917 req-5d055830-5a6b-4277-86af-a878aed609b8 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:16.570962 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-ee7e9605-acdf-435e-9196-297c1c0f2917 req-5d055830-5a6b-4277-86af-a878aed609b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:16.571456 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-ee7e9605-acdf-435e-9196-297c1c0f2917 req-5d055830-5a6b-4277-86af-a878aed609b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:16.583592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-ee7e9605-acdf-435e-9196-297c1c0f2917 req-5d055830-5a6b-4277-86af-a878aed609b8 service nova] Creating event network-vif-plugged:fefb02ab-67c7-4cc8-91df-d3860ccf795b for instance 5fb835d7-4e01-4735-9093-682fe2f848e0 on hv-cinder-zed-3 Oct 28 11:10:16.588699 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-ee7e9605-acdf-435e-9196-297c1c0f2917 req-5d055830-5a6b-4277-86af-a878aed609b8 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031590 Oct 28 11:10:16.589305 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 433/853] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:10:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:10:16.748567 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2338a6f5-70d9-4564-adef-e19825ca45db"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:10:16.751358 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:16.757865 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:16.758396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:16.758836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:16.759540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:16.759999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:16.760425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:16.997202 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-736d7d5f-4054-4277-bebf-dba0f9cbd597 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.253304 Oct 28 11:10:16.997804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 421/854] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:10:16 2022] POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-volume_attachments => generated 194 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:10:17.537306 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:17.537922 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:17.543945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:17.544420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:17.544848 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:17.545534 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:17.545998 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:17.546426 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:17.677182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:17.685099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-78adeb88-dc58-4ff7-8e1f-df8aafd7b68e tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.152083 Oct 28 11:10:17.685716 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 434/855] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:17 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:18.710294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:18.710989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:18.717173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:18.717655 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:18.718125 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:18.718811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:18.719258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:18.719676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:18.850341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:18.858044 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5be08965-517f-4b90-bc4f-c1e88ad9b454 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1266 microversion: 2.1 time: 0.153160 Oct 28 11:10:18.858801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 422/856] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:18 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:19.880153 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:19.880808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:19.886819 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:19.887296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:19.887729 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:19.888405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:19.888862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:19.889311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:20.018070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:20.026112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dbbb4ab2-b32b-4cc7-bb68-c08090c0e9ac tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1454 microversion: 2.1 time: 0.149819 Oct 28 11:10:20.026684 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 435/857] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:19 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1454 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:20.048388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:20.049006 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:20.055392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:20.055875 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:20.056311 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:20.056996 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:20.057448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:20.057872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:20.188637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:20.196992 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2514714f-49fc-43a0-9661-a354a096fe6c tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1454 microversion: 2.1 time: 0.152498 Oct 28 11:10:20.197652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 423/858] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:20 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1454 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:22.091964 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-90372e26-105e-486d-a239-e7a308e89054 req-f9fd3367-3645-4434-aa69-25b67380f07c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5fb835d7-4e01-4735-9093-682fe2f848e0", "tag": "fefb02ab-67c7-4cc8-91df-d3860ccf795b"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:10:22.102134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-90372e26-105e-486d-a239-e7a308e89054 req-f9fd3367-3645-4434-aa69-25b67380f07c service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:22.102607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-90372e26-105e-486d-a239-e7a308e89054 req-f9fd3367-3645-4434-aa69-25b67380f07c service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:22.103032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-90372e26-105e-486d-a239-e7a308e89054 req-f9fd3367-3645-4434-aa69-25b67380f07c service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:22.114352 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-90372e26-105e-486d-a239-e7a308e89054 req-f9fd3367-3645-4434-aa69-25b67380f07c service nova] Creating event network-changed:fefb02ab-67c7-4cc8-91df-d3860ccf795b for instance 5fb835d7-4e01-4735-9093-682fe2f848e0 on hv-cinder-zed-3 Oct 28 11:10:22.119521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-90372e26-105e-486d-a239-e7a308e89054 req-f9fd3367-3645-4434-aa69-25b67380f07c service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031703 Oct 28 11:10:22.120117 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 436/859] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:10:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:10:24.528078 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1abee8d3-f469-458f-a562-771636a1be58 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:10:24.534547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1abee8d3-f469-458f-a562-771636a1be58 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:24.535019 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1abee8d3-f469-458f-a562-771636a1be58 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:24.535473 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1abee8d3-f469-458f-a562-771636a1be58 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:24.595205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1abee8d3-f469-458f-a562-771636a1be58 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Going to try to stop instance {{(pid=120544) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 28 11:10:24.645236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1abee8d3-f469-458f-a562-771636a1be58 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action" status: 202 len: 0 microversion: 2.1 time: 0.120874 Oct 28 11:10:24.645850 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 424/860] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:10:24 2022] POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action => generated 0 bytes in 122 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:10:24.655508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:24.656220 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:24.662993 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:24.663468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:24.663893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:24.664575 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:24.665031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:24.665450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:24.793087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:24.801196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d94b0bf2-b41c-42cb-aab3-77e02d5d4005 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.150211 Oct 28 11:10:24.801764 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 437/861] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:24 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:10:25.825173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:25.825858 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:25.831937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:25.832411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:25.832835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:25.833540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:25.833998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:25.834418 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:25.964206 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:25.972205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7d5b5405-35c1-4e4c-a781-93521da5b671 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.151773 Oct 28 11:10:25.972765 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 425/862] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:25 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:26.998112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:26.998790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:27.004761 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:27.005235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:27.005662 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:27.006376 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:27.006858 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:27.007280 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:27.137024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:27.140586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=100 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:27.146035 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-24d90ba0-29bc-446d-a7cc-508715f74483 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.151984 Oct 28 11:10:27.146610 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 438/863] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:26 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:28.169093 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:28.169729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:28.176575 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:28.176766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:28.177197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:28.177913 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:28.178420 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:28.178848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:28.305136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:28.313281 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0b033773-f26b-417a-9711-84f5f12d1f83 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.148022 Oct 28 11:10:28.313848 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 426/864] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:28 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:29.336912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:29.337458 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:29.343439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:29.343909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:29.344334 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:29.345019 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:29.345468 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:29.345882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:29.473219 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:29.481836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dfb2936a-1f2f-4152-a8bb-4d0a343f3eab tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.149532 Oct 28 11:10:29.482432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 439/865] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:29 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:30.505283 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:30.506075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:30.513034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:30.513504 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:30.513929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:30.514635 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:30.515080 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:30.515496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:30.642489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:30.651181 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e84eb85f-0b1d-46aa-9c09-40ad21332223 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.149799 Oct 28 11:10:30.651751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 427/866] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:30 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:31.674163 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:31.674772 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:31.680727 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:31.681205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:31.681642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:31.682348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:31.682798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:31.683218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:31.810461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:31.818947 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4bb7f6a6-c644-4b7e-a0b9-8ea1b566cb89 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.148666 Oct 28 11:10:31.819561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 440/867] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:31 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:32.842202 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:32.842810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:32.849169 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:32.849640 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:32.850132 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:32.850810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:32.851252 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:32.851663 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:32.979602 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:32.981923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:32.990241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-48dd3705-a393-49b1-be33-060d07854f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.151922 Oct 28 11:10:32.990797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 428/868] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:32 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:34.013369 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:34.013896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:34.019877 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:34.020432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:34.020935 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:34.021680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:34.022237 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:34.022746 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:34.149550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:34.157688 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1019a8f8-5630-431b-a16d-8a76c7b9b3e5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.148300 Oct 28 11:10:34.158308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 441/869] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:34 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:35.181134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:35.181772 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:35.184449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=38 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:35.190568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:35.191045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:35.191476 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:35.192156 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:35.192603 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:35.193021 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:35.321124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:35.329165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f788c0d2-82cf-4d60-ae92-2862998c1501 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.151949 Oct 28 11:10:35.329729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 429/870] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:35 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:36.353314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:36.353950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:36.360007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:36.360478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:36.360906 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:36.361583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:36.362059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:36.362483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:36.490479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:36.499057 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f5f64f8c-ea57-4b30-8383-0395bf1c6ec1 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.150242 Oct 28 11:10:36.499626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 442/871] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:36 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:37.521741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:37.523006 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:37.529340 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:37.529817 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:37.530288 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:37.530966 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:37.531421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:37.531838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:37.658583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:37.667264 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-43cf46e3-5c4d-4c5b-b377-91e2e81b4e54 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.149325 Oct 28 11:10:37.667865 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 430/872] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:37 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:38.691554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:38.692165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:38.694049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=38 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:10:38.699452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:38.699929 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:38.700361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:38.701043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:38.701491 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:38.701909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:38.827642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:38.836526 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cbedae5e-1f17-4e5e-9c81-b79d345f13ba tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.148851 Oct 28 11:10:38.837096 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 443/873] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:38 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:39.858734 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:39.859345 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:39.866562 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:39.867040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:39.867471 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:39.868151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:39.868614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:39.869016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:39.992871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:40.001007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d41a6712-7f24-4aaa-8f5a-a32fcdab7f44 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.146044 Oct 28 11:10:40.001575 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 431/874] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:39 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:41.024514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:41.025124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:41.031120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:41.031599 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:41.032036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:41.032725 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:41.033185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:41.033608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:41.161425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:41.170394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-036301c9-c4d0-4c84-aec5-5a012e792737 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.149719 Oct 28 11:10:41.170974 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 444/875] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:41 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:42.194157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:42.194757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:42.201084 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:42.201560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:42.202014 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:42.202696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:42.203146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:42.203565 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:42.334355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:42.342933 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-59a67a69-462e-49e3-a0ce-0b4702855d65 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.152709 Oct 28 11:10:42.343496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 432/876] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:42 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:43.366916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:43.367522 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:43.373470 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:43.373946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:43.374405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:43.375087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:43.375533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:43.375952 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:43.499599 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:43.508275 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e0c06d86-fdad-4a8a-93d3-0d704a3ac777 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.145525 Oct 28 11:10:43.508846 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 445/877] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:43 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:44.532221 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:44.532748 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:44.539681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:44.540150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:44.540576 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:44.541246 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:44.541692 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:44.542150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:44.666344 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:44.674460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6f866132-5a3c-4e61-a2e0-e8528cb25aec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1721 microversion: 2.1 time: 0.146181 Oct 28 11:10:44.675023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 433/878] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:44 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1721 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:45.697375 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:45.698129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:45.704188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:45.704673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:45.705109 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:45.705818 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:45.706300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:45.706727 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:45.833256 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:45.841562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fc6bcc1a-e38e-48c3-9356-3810ebb20785 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1698 microversion: 2.1 time: 0.148010 Oct 28 11:10:45.842173 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 446/879] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:45 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1698 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:45.864392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a3041c8a-6deb-4f30-85f5-555aa07e8179 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:10:45.871226 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3041c8a-6deb-4f30-85f5-555aa07e8179 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:45.871710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3041c8a-6deb-4f30-85f5-555aa07e8179 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:45.872148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a3041c8a-6deb-4f30-85f5-555aa07e8179 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:45.894201 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a3041c8a-6deb-4f30-85f5-555aa07e8179 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Going to try to start instance {{(pid=120544) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 28 11:10:45.934802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a3041c8a-6deb-4f30-85f5-555aa07e8179 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action" status: 202 len: 0 microversion: 2.1 time: 0.074210 Oct 28 11:10:45.935658 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 434/880] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:10:45 2022] POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:10:45.943836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:45.944483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:45.950464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:45.950869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:45.951301 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:45.951993 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:45.952449 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:45.952866 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:46.120227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:46.128278 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c2bdea05-e575-48c0-964d-f11a497b9226 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1707 microversion: 2.1 time: 0.188225 Oct 28 11:10:46.128845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 447/881] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:45 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1707 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:47.151720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:47.152412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:47.159034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:47.159521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:47.159987 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:47.160747 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:47.161294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:47.161760 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:47.288563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:47.296702 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6cdd4349-d424-4bb5-a11b-0e5af6737d07 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1707 microversion: 2.1 time: 0.149882 Oct 28 11:10:47.297263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 435/882] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:47 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1707 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:10:48.320268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:10:48.320903 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:10:48.326904 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:48.327380 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:48.327813 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:48.328490 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:10:48.328940 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:10:48.329358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:10:48.456002 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:10:48.464124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4c7991b9-322c-427e-a276-92e227fa5fd0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1711 microversion: 2.1 time: 0.148378 Oct 28 11:10:48.464690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 448/883] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:10:48 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1711 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:11.840099 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:11.842359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:11.844696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=13 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:11.845533 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=47,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:11.851520 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:11.852005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:11.852441 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:11.853127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:11.853596 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:11.854077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:11.886215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-191bf80e-6eea-495d-b38e-916083629ed9 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.050662 Oct 28 11:11:11.886770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 436/884] 10.222.0.46 () {60 vars in 1343 bytes} [Fri Oct 28 11:11:11 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:11.949303 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:11.949798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:11.951832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=11 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:11.952501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=60 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:11.958227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:11.958715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:11.959149 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:11.959868 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:11.960325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:11.960748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:11.986317 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:11:12.058691 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3c8bab7d-b954-440d-b27d-192b850bf984 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 204 len: 0 microversion: 2.20 time: 0.113381 Oct 28 11:11:12.059696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 449/885] 10.222.0.46 () {60 vars in 1280 bytes} [Fri Oct 28 11:11:11 2022] DELETE /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 28 11:11:12.068082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:12.068719 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:12.074544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:12.075015 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:12.075442 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:12.076119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:12.076563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:12.076977 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:12.356476 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:12.364700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:12.366372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:12.368261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-469531bc-9cc1-4ef7-95ea-7e3cbe3ada3b tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.303584 Oct 28 11:11:12.368824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 437/886] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:11:12 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:11:13.394353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:13.395721 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:13.401864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:13.402368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:13.402791 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:13.403465 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:13.403909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:13.404322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:13.532086 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:13.540143 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:13.541565 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:13.542732 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3ea661bb-6597-4dd8-903f-bba755af9638 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1721 microversion: 2.20 time: 0.152740 Oct 28 11:11:13.543290 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 450/887] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:11:13 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:11:14.520305 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-3fed354a-371f-4531-8239-a918ed8345a9 req-7d3348be-6d32-4665-810a-56d450fb73dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "057e4fde-a7eb-419f-b0ba-ad3e183bf191", "name": "network-vif-deleted", "tag": "ded90a4e-c044-4824-8ed3-a7277034c619"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:11:14.531241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-3fed354a-371f-4531-8239-a918ed8345a9 req-7d3348be-6d32-4665-810a-56d450fb73dc service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:14.531778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-3fed354a-371f-4531-8239-a918ed8345a9 req-7d3348be-6d32-4665-810a-56d450fb73dc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:14.532205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-3fed354a-371f-4531-8239-a918ed8345a9 req-7d3348be-6d32-4665-810a-56d450fb73dc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:14.544236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-3fed354a-371f-4531-8239-a918ed8345a9 req-7d3348be-6d32-4665-810a-56d450fb73dc service nova] Creating event network-vif-deleted:ded90a4e-c044-4824-8ed3-a7277034c619 for instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 on hv-cinder-zed-3 Oct 28 11:11:14.549532 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-3fed354a-371f-4531-8239-a918ed8345a9 req-7d3348be-6d32-4665-810a-56d450fb73dc service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033824 Oct 28 11:11:14.550310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 438/888] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:11:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:11:14.567680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:14.568981 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:14.583583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:14.584493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:14.585128 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:14.586139 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:14.586941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:14.587370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:14.660918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:14.672359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:14.674539 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fc81231c965947ecaced8013076edafc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '312745d2566343faa50fa7c6f06483f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:14.676051 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2b6af4e3-3bd7-4897-8d52-9158c2d2d4a3 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 200 len: 1365 microversion: 2.20 time: 0.112094 Oct 28 11:11:14.677167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 451/889] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:11:14 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 1365 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 28 11:11:15.705901 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:15.706565 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:15.712690 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:15.713164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:15.713597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:15.714312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:15.714766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:15.715209 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:15.728628 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] HTTP exception thrown: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Request method failure captured: Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 HTTP/1.1 Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aHsv-Cx86mKuAI5Shg18AlPR6xZ9sSG68gjEeMNN4QBBuvVubjojiM-maRWCHJA1I-qtZUJi4kK3Mvi4JLVnFqpkrm_amShPb9gVMOrBf0R7ZOATcoqPjow565XoI6Yc4KKY4UR7UlVW9l1xhNOeSpJyMZIN6xBsGQYwhEPyYJIPI Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:11:15.730272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_312745d2566343faa50fa7c6f06483f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: fc81231c965947ecaced8013076edafc Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:15.732050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '057e4fde-a7eb-419f-b0ba-ad3e183bf191'} Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:11:15.732975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:15.734664 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:15.736339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Returning 404 to user: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:11:15.736339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b081d31e-6d91-437d-9fa4-5dcf5eda4706 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 404 len: 111 microversion: 2.20 time: 0.030527 Oct 28 11:11:15.736339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 439/890] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:11:15 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:11:15.740782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ae34ccb5-c898-441a-b653-f5c0bc7575ea tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '.version_select at 0x7f42fa4ae160>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:15.746083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR oslo.messaging._drivers.impl_rabbit [None req-ae34ccb5-c898-441a-b653-f5c0bc7575ea tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [b925a8e9-20ed-4e25-9474-778da0105f3b] AMQP server on 10.222.0.46:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 28 11:11:16.753160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO oslo.messaging._drivers.impl_rabbit [None req-ae34ccb5-c898-441a-b653-f5c0bc7575ea tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [b925a8e9-20ed-4e25-9474-778da0105f3b] Reconnected to AMQP server on 10.222.0.46:5672 via [amqp] client with port 34796. Oct 28 11:11:16.771714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ae34ccb5-c898-441a-b653-f5c0bc7575ea tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-235302277" status: 204 len: 0 microversion: 2.20 time: 1.035026 Oct 28 11:11:16.772402 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 452/891] 10.222.0.46 () {60 vars in 1259 bytes} [Fri Oct 28 11:11:15 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-235302277 => generated 0 bytes in 1036 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 28 11:11:18.742563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:18.743705 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:18.750398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:18.750874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:18.751302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:18.751985 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:18.752433 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:18.752849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:18.765723 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] HTTP exception thrown: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Request method failure captured: Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: DELETE /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 HTTP/1.1 Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aHsv-Cx86mKuAI5Shg18AlPR6xZ9sSG68gjEeMNN4QBBuvVubjojiM-maRWCHJA1I-qtZUJi4kK3Mvi4JLVnFqpkrm_amShPb9gVMOrBf0R7ZOATcoqPjow565XoI6Yc4KKY4UR7UlVW9l1xhNOeSpJyMZIN6xBsGQYwhEPyYJIPI Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:11:18.767275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_312745d2566343faa50fa7c6f06483f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: fc81231c965947ecaced8013076edafc Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.769016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '057e4fde-a7eb-419f-b0ba-ad3e183bf191'} Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:11:18.769935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.771445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.773286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Returning 404 to user: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:11:18.773286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-285de5ca-ad13-4ed9-8248-4fd012781b40 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 404 len: 111 microversion: 2.20 time: 0.030159 Oct 28 11:11:18.773286 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 440/892] 10.222.0.46 () {60 vars in 1280 bytes} [Fri Oct 28 11:11:18 2022] DELETE /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:11:18.777276 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:18.777891 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: 057e4fde-a7eb-419f-b0ba-ad3e183bf191] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:18.784569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:18.785051 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:18.785488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:18.786204 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:18.786666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:18.787099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:18.800338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] HTTP exception thrown: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Request method failure captured: Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 HTTP/1.1 Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7aHsv-Cx86mKuAI5Shg18AlPR6xZ9sSG68gjEeMNN4QBBuvVubjojiM-maRWCHJA1I-qtZUJi4kK3Mvi4JLVnFqpkrm_amShPb9gVMOrBf0R7ZOATcoqPjow565XoI6Yc4KKY4UR7UlVW9l1xhNOeSpJyMZIN6xBsGQYwhEPyYJIPI Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:11:18.801996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_312745d2566343faa50fa7c6f06483f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: fc81231c965947ecaced8013076edafc Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.803621 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '057e4fde-a7eb-419f-b0ba-ad3e183bf191'} Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:11:18.808085 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. Oct 28 11:11:18.809549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:18.811551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Returning 404 to user: Instance 057e4fde-a7eb-419f-b0ba-ad3e183bf191 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:11:18.811551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c0e96d64-022e-4cc1-9b7c-c65e6890e366 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191" status: 404 len: 111 microversion: 2.20 time: 0.030036 Oct 28 11:11:18.811551 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 453/893] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:11:18 2022] GET /compute/v2.1/servers/057e4fde-a7eb-419f-b0ba-ad3e183bf191 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:11:19.944382 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:19.944888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:19.951797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:19.952272 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:19.952704 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:19.953383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:19.953833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:19.954392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:19.967172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] HTTP exception thrown: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Request method failure captured: Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: DELETE /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a HTTP/1.1 Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7aHsv-Cx86mKuAI5Shg18AlPR6xZ9sSG68gjEeMNN4QBBuvVubjojiM-maRWCHJA1I-qtZUJi4kK3Mvi4JLVnFqpkrm_amShPb9gVMOrBf0R7ZOATcoqPjow565XoI6Yc4KKY4UR7UlVW9l1xhNOeSpJyMZIN6xBsGQYwhEPyYJIPI Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:11:19.968695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_312745d2566343faa50fa7c6f06483f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: fc81231c965947ecaced8013076edafc Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:19.970327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'e4ef9cf5-ad72-4399-a910-9308a243a57a'} Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:11:19.971373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:19.972833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:11:19.974417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Returning 404 to user: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:11:19.974417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6e30675f-e5f0-49f6-9cb1-9936aa717b59 tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "DELETE /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 404 len: 111 microversion: 2.20 time: 0.029526 Oct 28 11:11:19.974417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 441/894] 10.222.0.46 () {60 vars in 1280 bytes} [Fri Oct 28 11:11:19 2022] DELETE /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:11:19.979049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:19.979660 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] [instance: e4ef9cf5-ad72-4399-a910-9308a243a57a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:19.985488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:19.985958 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:19.986437 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:19.987116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:19.987563 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:19.987981 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:20.001585 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] HTTP exception thrown: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Request method failure captured: Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a HTTP/1.1 Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7aHsv-Cx86mKuAI5Shg18AlPR6xZ9sSG68gjEeMNN4QBBuvVubjojiM-maRWCHJA1I-qtZUJi4kK3Mvi4JLVnFqpkrm_amShPb9gVMOrBf0R7ZOATcoqPjow565XoI6Yc4KKY4UR7UlVW9l1xhNOeSpJyMZIN6xBsGQYwhEPyYJIPI Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Openstack-Nova-Api-Version: 2.20 Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:11:20.003127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_312745d2566343faa50fa7c6f06483f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/312745d2566343faa50fa7c6f06483f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 312745d2566343faa50fa7c6f06483f1 Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1894077199 Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: fc81231c965947ecaced8013076edafc Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1894077199-project Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:20.004737 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'e4ef9cf5-ad72-4399-a910-9308a243a57a'} Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:11:20.005874 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. Oct 28 11:11:20.007363 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:11:20.008828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] Returning 404 to user: Instance e4ef9cf5-ad72-4399-a910-9308a243a57a could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:11:20.008828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7b70be37-f8a6-455a-8233-68a08d409dde tempest-AttachVolumeShelveTestJSON-1894077199 tempest-AttachVolumeShelveTestJSON-1894077199-project] 10.222.0.46 "GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a" status: 404 len: 111 microversion: 2.20 time: 0.029888 Oct 28 11:11:20.008828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 454/895] 10.222.0.46 () {60 vars in 1277 bytes} [Fri Oct 28 11:11:19 2022] GET /compute/v2.1/servers/e4ef9cf5-ad72-4399-a910-9308a243a57a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:11:28.300310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c1615ed7-9859-4264-8f0c-73c8623cde48 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: .version_select at 0x7f42fa5f1d30>, body: {"keypair": {"name": "tempest-keypair-1930240948", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMIS07AOO+rDLc7peoVwUMb6kjJ8TcdnAB/xWZCa+Ye5wkCfJS8MKzFPLGK7H3xHqVZV4Qokvxq9aBhZy/ySU7+NWd5Ni7KLR/tkSpWm2LEFvzbZYbLLhj6IIDoN5cD37g=="}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:11:28.316768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-c1615ed7-9859-4264-8f0c-73c8623cde48 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:11:28.321830 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-c1615ed7-9859-4264-8f0c-73c8623cde48 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:11:28.334327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR oslo.messaging._drivers.impl_rabbit [None req-c1615ed7-9859-4264-8f0c-73c8623cde48 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [935c4775-2880-4fea-93df-43580021b5eb] AMQP server on 10.222.0.46:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 28 11:11:29.342527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO oslo.messaging._drivers.impl_rabbit [None req-c1615ed7-9859-4264-8f0c-73c8623cde48 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [935c4775-2880-4fea-93df-43580021b5eb] Reconnected to AMQP server on 10.222.0.46:5672 via [amqp] client with port 34888. Oct 28 11:11:29.357801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c1615ed7-9859-4264-8f0c-73c8623cde48 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.217319 Oct 28 11:11:29.358724 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 442/896] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:11:28 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1219 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:11:30.880421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-338276414", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "447754b6-a798-4b6b-beaa-8e9ec6ea9fd5"}], "security_groups": [{"name": "tempest-securitygroup--1577325693"}], "key_name": "tempest-keypair-1930240948", "user_data": "}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:11:30.891887 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:30.892552 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:31.099099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] validate_networks() for [('447754b6-a798-4b6b-beaa-8e9ec6ea9fd5', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:11:31.476273 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:11:31.476692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:11:31.477728 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:11:31.478166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:11:31.478598 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:11:31.487331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:31.487812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:31.488241 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:31.489215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:31.489669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:31.490127 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:31.507131 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:11:31.511696 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:11:31.522464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:11:31.525310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:11:31.573366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a1a1a583-e9f9-4d08-ac37-3278adad2659 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.697784 Oct 28 11:11:31.574249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 455/897] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:11:30 2022] POST /compute/v2.1/servers => generated 402 bytes in 699 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:11:31.581882 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=20 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:31.586701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f2b8e2f2-db3d-4809-9c96-eb8aabe13e0c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:31.587310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f2b8e2f2-db3d-4809-9c96-eb8aabe13e0c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:31.699730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f2b8e2f2-db3d-4809-9c96-eb8aabe13e0c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:31.708928 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f2b8e2f2-db3d-4809-9c96-eb8aabe13e0c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1241 microversion: 2.1 time: 0.127167 Oct 28 11:11:31.709399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 443/898] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:31 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1241 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:32.730762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:32.731460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:32.738550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:32.739099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:32.739540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:32.740233 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:32.740686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:32.741102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:32.818876 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:32.827661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-122dd944-d8fb-4947-b3b2-2fe377724c28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1301 microversion: 2.1 time: 0.100825 Oct 28 11:11:32.828336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 456/899] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:32 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1301 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:33.851051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:33.851699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:33.858379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:33.859076 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:33.859512 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:33.860241 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:33.860696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:33.861117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:33.996105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:34.003696 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1542fe50-802f-498b-baeb-b80d0344bf02 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.156404 Oct 28 11:11:34.004257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 444/900] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:33 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:34.199345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-5d21ca73-8dba-4b13-ae1d-3c35cbffd80b req-e8414534-0480-4deb-9e21-119da89b89da service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "952a161f-410d-4266-b236-14823a631423", "tag": "da22476d-00ea-4b57-aedf-ea0bfc3eaaa2"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:11:34.211079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-5d21ca73-8dba-4b13-ae1d-3c35cbffd80b req-e8414534-0480-4deb-9e21-119da89b89da service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:34.224217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-5d21ca73-8dba-4b13-ae1d-3c35cbffd80b req-e8414534-0480-4deb-9e21-119da89b89da service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:34.224846 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-5d21ca73-8dba-4b13-ae1d-3c35cbffd80b req-e8414534-0480-4deb-9e21-119da89b89da service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:34.244475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-5d21ca73-8dba-4b13-ae1d-3c35cbffd80b req-e8414534-0480-4deb-9e21-119da89b89da service nova] Creating event network-changed:da22476d-00ea-4b57-aedf-ea0bfc3eaaa2 for instance 952a161f-410d-4266-b236-14823a631423 on hv-cinder-zed-3 Oct 28 11:11:34.260875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-5d21ca73-8dba-4b13-ae1d-3c35cbffd80b req-e8414534-0480-4deb-9e21-119da89b89da service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066070 Oct 28 11:11:34.266731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 457/901] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:11:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:11:35.028247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:35.028863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:35.035331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:35.035810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:35.036245 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:35.036927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:35.037380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:35.037804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:35.162163 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:35.169663 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-80147b92-6a36-4bba-b2e6-f2ad6402c875 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.146840 Oct 28 11:11:35.170252 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 445/902] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:35 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:36.192834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:36.193448 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:36.200122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:36.200597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:36.201028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:36.201705 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:36.202186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:36.202609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:36.328951 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:36.336596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b47ea527-7932-486e-9b6a-271356058886 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.148071 Oct 28 11:11:36.337164 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 458/903] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:36 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:37.358659 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:37.360018 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:37.367269 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:37.367786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:37.368265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:37.368955 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:37.369403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:37.369851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:37.510025 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:37.517906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-625f6042-61f6-4663-b74e-398857e8b5c4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.163512 Oct 28 11:11:37.528681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 446/904] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:37 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:38.541475 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:38.542050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:38.548310 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:38.548778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:38.549211 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:38.549893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:38.550377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:38.551032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:38.680828 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:38.688351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ca82a4bf-710e-4b3f-89cf-f85ac87583a4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.151426 Oct 28 11:11:38.688919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 459/905] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:38 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:39.711583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:39.712245 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:39.718698 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:39.719175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:39.719605 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:39.720284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:39.720735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:39.721153 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:39.851173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:39.859543 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-021326c8-222d-4357-b877-acf6c51a0abe tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.151875 Oct 28 11:11:39.860094 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 447/906] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:39 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:40.882818 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:40.883971 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:40.891292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:40.891780 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:40.892213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:40.892911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:40.893361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:40.893783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:41.022786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:41.032074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1d68a423-36d4-412f-aacf-8054266fc8fa tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.152930 Oct 28 11:11:41.032976 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 460/907] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:40 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:41.042329 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-27adcd84-2d95-4df9-a8d1-adaa79dc3b3a req-83cb6c2a-a39d-432f-9209-6a7a8debaa44 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "952a161f-410d-4266-b236-14823a631423", "name": "network-vif-plugged", "status": "completed", "tag": "da22476d-00ea-4b57-aedf-ea0bfc3eaaa2"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:11:41.054019 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-27adcd84-2d95-4df9-a8d1-adaa79dc3b3a req-83cb6c2a-a39d-432f-9209-6a7a8debaa44 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:41.054486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-27adcd84-2d95-4df9-a8d1-adaa79dc3b3a req-83cb6c2a-a39d-432f-9209-6a7a8debaa44 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:41.054940 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-27adcd84-2d95-4df9-a8d1-adaa79dc3b3a req-83cb6c2a-a39d-432f-9209-6a7a8debaa44 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:41.065799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-27adcd84-2d95-4df9-a8d1-adaa79dc3b3a req-83cb6c2a-a39d-432f-9209-6a7a8debaa44 service nova] Creating event network-vif-plugged:da22476d-00ea-4b57-aedf-ea0bfc3eaaa2 for instance 952a161f-410d-4266-b236-14823a631423 on hv-cinder-zed-3 Oct 28 11:11:41.072687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-27adcd84-2d95-4df9-a8d1-adaa79dc3b3a req-83cb6c2a-a39d-432f-9209-6a7a8debaa44 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034212 Oct 28 11:11:41.073455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 448/908] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:11:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:11:42.055390 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:42.056023 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:42.057999 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:42.058652 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=6 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:42.064562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:42.065041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:42.065467 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:42.066185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:42.066635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:42.067054 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:42.196167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:42.203673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4e0976a1-643e-4e35-a59a-aeae852ef76f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1367 microversion: 2.1 time: 0.152741 Oct 28 11:11:42.204242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 461/909] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:42 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:43.226766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:43.227419 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:43.229531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:43.230620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:43.236621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:43.237093 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:43.237519 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:43.238228 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:43.238938 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:43.239352 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:43.364881 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:43.372568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6c7f645f-aeff-487c-acc5-119090843f47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1556 microversion: 2.1 time: 0.150123 Oct 28 11:11:43.373116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 449/910] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:43 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1556 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:43.395106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:43.395615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:43.402032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:43.402518 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:43.402954 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:43.403634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:43.404088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:43.404510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:43.479836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c0329f82-8cac-4a72-9e38-7512e9cdbae8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.088491 Oct 28 11:11:43.480441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 462/911] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:11:43 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-interface => generated 284 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:11:44.626350 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-5458c53a-45d0-4a37-b7c6-4070d5a0c835 req-d05aaece-809f-4557-a312-a423cb084397 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "952a161f-410d-4266-b236-14823a631423", "tag": "da22476d-00ea-4b57-aedf-ea0bfc3eaaa2"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:11:44.636157 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-5458c53a-45d0-4a37-b7c6-4070d5a0c835 req-d05aaece-809f-4557-a312-a423cb084397 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:44.636621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-5458c53a-45d0-4a37-b7c6-4070d5a0c835 req-d05aaece-809f-4557-a312-a423cb084397 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:44.637043 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-5458c53a-45d0-4a37-b7c6-4070d5a0c835 req-d05aaece-809f-4557-a312-a423cb084397 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:44.647598 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-5458c53a-45d0-4a37-b7c6-4070d5a0c835 req-d05aaece-809f-4557-a312-a423cb084397 service nova] Creating event network-changed:da22476d-00ea-4b57-aedf-ea0bfc3eaaa2 for instance 952a161f-410d-4266-b236-14823a631423 on hv-cinder-zed-3 Oct 28 11:11:44.653144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-5458c53a-45d0-4a37-b7c6-4070d5a0c835 req-d05aaece-809f-4557-a312-a423cb084397 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030561 Oct 28 11:11:44.655512 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 450/912] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:11:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:11:55.635298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:55.636092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=38 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:55.640460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:55.640945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:55.649036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:55.649766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:55.650235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:55.651492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:55.651946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:55.652502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:55.732020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lazy-loading 'services' on Instance uuid e4432177-31d4-43c8-a6a2-a1d727b144a2 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:11:55.803080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-50fd065e-0df3-4c5f-a0a3-2268b8781f79 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-volume_attachments/2338a6f5-70d9-4564-adef-e19825ca45db" status: 202 len: 0 microversion: 2.1 time: 0.168496 Oct 28 11:11:55.803665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 463/913] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:11:55 2022] DELETE /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-volume_attachments/2338a6f5-70d9-4564-adef-e19825ca45db => generated 0 bytes in 170 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:11:56.925237 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:56.926022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=34 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:11:56.930371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e478758f-7772-4383-86c9-4d019a9132bb tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:11:56.937884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e478758f-7772-4383-86c9-4d019a9132bb tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:56.938386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e478758f-7772-4383-86c9-4d019a9132bb tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:56.938827 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e478758f-7772-4383-86c9-4d019a9132bb tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:56.959776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e478758f-7772-4383-86c9-4d019a9132bb tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Going to try to stop instance {{(pid=120544) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 28 11:11:56.999608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e478758f-7772-4383-86c9-4d019a9132bb tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action" status: 202 len: 0 microversion: 2.1 time: 0.075708 Oct 28 11:11:57.000189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 451/914] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:11:56 2022] POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:11:57.009275 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:57.010019 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:57.016488 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:57.017038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:57.017476 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:57.018203 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:57.018662 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:57.019086 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:57.149803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:57.158033 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1cde8e32-9d9f-444f-bd17-db4f5f2709a6 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.153114 Oct 28 11:11:57.158604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 464/915] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:57 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:58.182610 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:58.183306 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:58.189345 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:58.189811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:58.190265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:58.190939 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:58.191387 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:58.191800 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:58.320961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:58.329143 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-19d045ee-86dc-430b-ac25-776f4c7e81a8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.151343 Oct 28 11:11:58.329708 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 452/916] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:58 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:11:59.351833 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:11:59.352446 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:11:59.358979 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:59.359458 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:59.359889 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:59.360568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:11:59.361021 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:11:59.361441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:11:59.489122 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:11:59.496758 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2a95a30e-7ea7-4dae-9986-04b7b9aadfb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.148834 Oct 28 11:11:59.497328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 465/917] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:11:59 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:00.519681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:00.520220 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:00.527112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:00.527885 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:00.528382 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:00.528952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:00.529403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:00.529849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:00.665361 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:00.673066 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5d032633-8e82-4ff7-8a4e-07df63264e92 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.157315 Oct 28 11:12:00.673620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 453/918] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:00 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:01.695571 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:01.696209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:01.702634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:01.703118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:01.703586 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:01.704304 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:01.704758 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:01.705182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:01.829801 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:01.837460 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5432aea1-bb62-4fbd-ab8d-e1ff550f7146 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.145773 Oct 28 11:12:01.838043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 466/919] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:01 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:02.861489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:02.862131 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:02.868391 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:02.868872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:02.869309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:02.870607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:02.871065 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:02.871495 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:02.995999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:03.003614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dc2f21c6-5f8d-410d-80f7-b1db48281b0a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.146489 Oct 28 11:12:03.004171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 454/920] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:02 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:04.027307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:04.027855 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:04.033774 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:04.034285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:04.034719 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:04.035411 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:04.035860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:04.036283 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:04.161554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:04.169167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c1f7a80d-876b-49a4-85e1-be0db1225430 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.146344 Oct 28 11:12:04.169736 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 467/921] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:04 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:05.192361 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:05.192912 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:05.199838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:05.200313 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:05.200742 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:05.201421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:05.201868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:05.202310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:05.333414 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:05.341015 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f0977c77-de05-4aca-90bc-6d0014ff25c8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.152612 Oct 28 11:12:05.341573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 455/922] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:05 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:06.365252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:06.365862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:06.373040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:06.373517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:06.373948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:06.374672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:06.375121 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:06.375540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:06.502660 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:06.510175 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-10beb6ec-be2a-4717-be00-9f49927f6b19 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1675 microversion: 2.1 time: 0.149062 Oct 28 11:12:06.511435 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 468/923] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:06 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1675 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:07.533677 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:07.534248 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:07.540782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:07.541266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:07.541701 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:07.542929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:07.543384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:07.543807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:07.669322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:07.676999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-044690e3-e9ed-4a65-a7ea-c8f7e4666a8f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1652 microversion: 2.1 time: 0.147274 Oct 28 11:12:07.677589 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 456/924] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:07 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1652 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:07.700009 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ae78e855-451e-4ba9-8268-a4dd80844e4a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:07.706036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ae78e855-451e-4ba9-8268-a4dd80844e4a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:07.706531 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ae78e855-451e-4ba9-8268-a4dd80844e4a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:07.706963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ae78e855-451e-4ba9-8268-a4dd80844e4a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:07.766733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ae78e855-451e-4ba9-8268-a4dd80844e4a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Going to try to start instance {{(pid=120543) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 28 11:12:07.808593 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ae78e855-451e-4ba9-8268-a4dd80844e4a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action" status: 202 len: 0 microversion: 2.1 time: 0.111750 Oct 28 11:12:07.809174 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 469/925] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:12:07 2022] POST /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/action => generated 0 bytes in 114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:12:07.817512 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:07.818179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:07.824378 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:07.824859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:07.825392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:07.826152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:07.827336 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:07.827769 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:07.966230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:07.974119 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c5936834-d967-495f-934f-778f1523ec88 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1661 microversion: 2.1 time: 0.160403 Oct 28 11:12:07.974697 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 457/926] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:07 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1661 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:08.205550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:08.206202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:08.212222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:08.212706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:08.213146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:08.213843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:08.214845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:08.215265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:08.238433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lazy-loading 'flavor' on Instance uuid 8c0936af-77e0-4b5c-b84c-08f6f5d72735 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:12:08.312585 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f0896082-6738-4190-8371-18d55737c8eb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "POST /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735/action" status: 202 len: 0 microversion: 2.1 time: 0.111209 Oct 28 11:12:08.313178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 470/927] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:12:08 2022] POST /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735/action => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:12:08.321269 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:08.321879 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:08.329150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:08.329666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:08.330243 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:08.330971 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:08.331441 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:08.331964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:08.461571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:08.471240 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-81fe142c-e15f-4f76-9a58-02d1f7e3a47d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.153744 Oct 28 11:12:08.471800 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 458/928] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:08 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:08.998439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:08.999132 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:09.005202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:09.005681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:09.006150 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:09.006830 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:09.007280 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:09.007700 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:09.134902 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:09.142997 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d72ad65b-0ef1-483f-af5b-8fe5f593d34a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1661 microversion: 2.1 time: 0.148754 Oct 28 11:12:09.143597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 471/929] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:08 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1661 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:09.494499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:09.495108 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:09.502045 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:09.502526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:09.502956 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:09.503631 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:09.504097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:09.504499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:09.634560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:09.642920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-bb396e4a-1156-430f-9303-3c90b679ef60 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.152265 Oct 28 11:12:09.643478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 459/930] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:09 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:10.166566 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:10.167175 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:10.173020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:10.173496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:10.173927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:10.174629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:10.175083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:10.175501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:10.305279 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:10.313463 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5cdeed8e-2da2-40ae-877d-8b1b6833e5d3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1665 microversion: 2.1 time: 0.150815 Oct 28 11:12:10.314041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 472/931] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:10 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1665 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:10.666408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:10.667002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:10.673417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:10.673887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:10.674344 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:10.675032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:10.675480 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:10.675896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:10.802795 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:10.810431 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-980e043d-a1b5-4a52-b34b-f5372b7dc0ef tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.148388 Oct 28 11:12:10.810991 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 460/932] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:10 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:11.834377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:11.834984 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:11.841463 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:11.841921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:11.842376 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:11.843055 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:11.843503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:11.843917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:11.931891 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:11.932403 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:11.945064 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:11.945566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:11.946397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:11.947103 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:11.947771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:11.948191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:11.976676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:12:12.004847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:12.013908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-bdbd7fff-69a5-44f0-8480-35d58c4e4953 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.183156 Oct 28 11:12:12.014345 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 473/933] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:11 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:12.075911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-00b6de7c-5bed-440d-a321-42c8be1be1d0 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "DELETE /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 204 len: 0 microversion: 2.1 time: 0.147654 Oct 28 11:12:12.077048 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 461/934] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:12:11 2022] DELETE /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:12:12.088350 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:12.089822 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:12.096162 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-dfd64552-37bb-4514-b92c-5bd570c89d32 req-21f77d93-d1bf-4db0-ad13-b85f4faed7ab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5fb835d7-4e01-4735-9093-682fe2f848e0", "tag": "fefb02ab-67c7-4cc8-91df-d3860ccf795b"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:12.097947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:12.098523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:12.099106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:12.099811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:12.100259 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:12.100682 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:12.108083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-dfd64552-37bb-4514-b92c-5bd570c89d32 req-21f77d93-d1bf-4db0-ad13-b85f4faed7ab service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:12.108809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-dfd64552-37bb-4514-b92c-5bd570c89d32 req-21f77d93-d1bf-4db0-ad13-b85f4faed7ab service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:12.109401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-dfd64552-37bb-4514-b92c-5bd570c89d32 req-21f77d93-d1bf-4db0-ad13-b85f4faed7ab service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:12.123896 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-dfd64552-37bb-4514-b92c-5bd570c89d32 req-21f77d93-d1bf-4db0-ad13-b85f4faed7ab service nova] Creating event network-changed:fefb02ab-67c7-4cc8-91df-d3860ccf795b for instance 5fb835d7-4e01-4735-9093-682fe2f848e0 on hv-cinder-zed-3 Oct 28 11:12:12.130271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-dfd64552-37bb-4514-b92c-5bd570c89d32 req-21f77d93-d1bf-4db0-ad13-b85f4faed7ab service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.114596 Oct 28 11:12:12.131077 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 474/935] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:12:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:12:12.235228 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:12.244498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1b165973-2fbf-403e-9808-722fe93e18cc tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1578 microversion: 2.1 time: 0.162863 Oct 28 11:12:12.246920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 462/936] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:12 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1578 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:13.038189 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:13.038883 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:13.044945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:13.045423 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:13.045857 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:13.046559 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:13.047014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:13.047435 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:13.257452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:13.266151 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0e081526-3f2c-4093-b989-6aaeb8351495 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.232079 Oct 28 11:12:13.266728 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 475/937] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:13 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:13.276617 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:13.280141 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:13.290257 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:13.290741 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:13.291178 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:13.291865 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:13.292332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:13.292747 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:13.426824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:13.435023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-40f01637-54cc-4a2d-a9bd-cf9ed7fd225b tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1578 microversion: 2.1 time: 0.170330 Oct 28 11:12:13.435578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 463/938] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:13 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1578 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:14.288859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:14.289457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-a6b61920-e10b-4851-9c9b-56a280373586 req-8579e2ef-4305-405d-affb-afa9d3ba31f6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5fb835d7-4e01-4735-9093-682fe2f848e0", "name": "network-vif-deleted", "tag": "fefb02ab-67c7-4cc8-91df-d3860ccf795b"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:14.289835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:14.296205 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:14.296775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:14.297275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:14.298053 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:14.298724 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:14.299051 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-a6b61920-e10b-4851-9c9b-56a280373586 req-8579e2ef-4305-405d-affb-afa9d3ba31f6 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:14.299256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:14.299527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-a6b61920-e10b-4851-9c9b-56a280373586 req-8579e2ef-4305-405d-affb-afa9d3ba31f6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:14.299952 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-a6b61920-e10b-4851-9c9b-56a280373586 req-8579e2ef-4305-405d-affb-afa9d3ba31f6 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:14.311212 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-a6b61920-e10b-4851-9c9b-56a280373586 req-8579e2ef-4305-405d-affb-afa9d3ba31f6 service nova] Creating event network-vif-deleted:fefb02ab-67c7-4cc8-91df-d3860ccf795b for instance 5fb835d7-4e01-4735-9093-682fe2f848e0 on hv-cinder-zed-3 Oct 28 11:12:14.317099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-a6b61920-e10b-4851-9c9b-56a280373586 req-8579e2ef-4305-405d-affb-afa9d3ba31f6 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098692 Oct 28 11:12:14.317697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 476/939] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:12:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:12:14.435833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:14.439363 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:12:14.444557 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f95b1360-4227-448d-9eba-cd9a69d3d83d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.159732 Oct 28 11:12:14.445294 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 464/940] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:14 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:12:14.459252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:14.460267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:14.467387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:14.467900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:14.468339 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:14.469036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:14.470642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:14.470642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:14.474100 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:12:14.537485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:14.546638 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-6dfabe4b-1be7-4875-a78e-dc000c74cd82 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 200 len: 1225 microversion: 2.1 time: 0.091397 Oct 28 11:12:14.547232 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 477/941] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:14 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:15.468056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:15.468698 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:15.476130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:15.476607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:15.477038 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:15.477720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:15.478198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:15.478620 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:15.568615 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:15.569427 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 5fb835d7-4e01-4735-9093-682fe2f848e0] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:15.576218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:15.576780 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:15.577382 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:15.578214 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:15.578753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:15.579253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:15.592485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] HTTP exception thrown: Instance 5fb835d7-4e01-4735-9093-682fe2f848e0 could not be found. Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Request method failure captured: Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 HTTP/1.1 Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7bpKhsLly_AsLC5I5Ufi7RRtWwekiZ20oTVh9O4umzrCT6TaV8PH5o_R9SYBAOVP2EofrgrO2OiTcdq11VWJ9GiBna5aI-gXnS1_CYhIzgxrlZivt_OgETCTDDtqo0G4RtPvy7n959ltOqSgWxONV0J0lU0ZrvO6vH8e0oBtYQ2BtI Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:12:15.594251 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 6b926f7161c0487a8c2b04ada3b374e3 Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 5fb835d7-4e01-4735-9093-682fe2f848e0 could not be found. Oct 28 11:12:15.595945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '5fb835d7-4e01-4735-9093-682fe2f848e0'} Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5fb835d7-4e01-4735-9093-682fe2f848e0 could not be found. Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:12:15.597520 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5fb835d7-4e01-4735-9093-682fe2f848e0 could not be found. Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5fb835d7-4e01-4735-9093-682fe2f848e0 could not be found. Oct 28 11:12:15.599269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:15.600911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Returning 404 to user: Instance 5fb835d7-4e01-4735-9093-682fe2f848e0 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:12:15.600911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0aa6302f-bc5d-4e36-a530-7b8ff437b72d tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0" status: 404 len: 111 microversion: 2.1 time: 0.030878 Oct 28 11:12:15.600911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 478/942] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:15 2022] GET /compute/v2.1/servers/5fb835d7-4e01-4735-9093-682fe2f848e0 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 11:12:15.608142 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:15.617401 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4482ddc8-f9bf-43c3-8dec-b96489860e39 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.153228 Oct 28 11:12:15.617963 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 465/943] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:15 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:16.641246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:16.641877 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:16.647862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:16.648335 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:16.648765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:16.649441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:16.649889 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:16.650348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:16.783177 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:16.792684 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c9b9e62a-2db1-468a-992a-65a04cbe742f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.155311 Oct 28 11:12:16.793248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 479/944] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:16 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:17.815753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:17.817673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:17.825071 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:17.825492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:17.825927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:17.826643 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:17.827091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:17.827536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:17.963581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:17.971835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ff588b6e-2dd0-4747-9332-9b7cc6a6b762 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.159903 Oct 28 11:12:17.972420 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 466/945] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:17 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:18.921961 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-a649af51-c60f-458e-9322-5b146dce03c8 req-f0240d30-bcbd-46bf-82e0-de016eb153ae service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a", "tag": "77a42644-3142-4fb2-9d0e-5deb0792310a"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:18.923898 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:18.924925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:18.934545 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a649af51-c60f-458e-9322-5b146dce03c8 req-f0240d30-bcbd-46bf-82e0-de016eb153ae service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:18.934797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a649af51-c60f-458e-9322-5b146dce03c8 req-f0240d30-bcbd-46bf-82e0-de016eb153ae service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:18.935180 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:18.935316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a649af51-c60f-458e-9322-5b146dce03c8 req-f0240d30-bcbd-46bf-82e0-de016eb153ae service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:18.935876 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:18.936318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:18.937115 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:18.937872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:18.938508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:18.950930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-a649af51-c60f-458e-9322-5b146dce03c8 req-f0240d30-bcbd-46bf-82e0-de016eb153ae service nova] Creating event network-changed:77a42644-3142-4fb2-9d0e-5deb0792310a for instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a on hv-cinder-zed-3 Oct 28 11:12:18.956647 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-a649af51-c60f-458e-9322-5b146dce03c8 req-f0240d30-bcbd-46bf-82e0-de016eb153ae service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038178 Oct 28 11:12:18.958016 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 467/946] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:12:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:12:18.972249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:12:18.999204 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:19.000389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:19.013672 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:19.014617 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:19.015423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:19.016560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:19.017356 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:19.018334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:19.074492 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-00910002-e3f3-41cf-900d-bfb547c74558 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "DELETE /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 204 len: 0 microversion: 2.1 time: 0.157485 Oct 28 11:12:19.075038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 480/947] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:12:18 2022] DELETE /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:12:19.090960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:19.094199 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:19.107747 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:19.108242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:19.108680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:19.109390 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:19.109852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:19.110299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:19.220022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:19.229050 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-29f2bef8-9761-4c20-9c53-5b8a65310ccf tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.235733 Oct 28 11:12:19.244746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 468/948] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:18 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:19.267597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:19.276040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c01186be-fdac-479d-b073-cba0aac007b7 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1579 microversion: 2.1 time: 0.195907 Oct 28 11:12:19.277164 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 481/949] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:19 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1579 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:20.252863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:20.253405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:20.261626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:20.262151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:20.262584 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:20.263344 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:20.263778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:20.264186 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:20.300473 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:20.301182 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:20.307298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:20.307888 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:20.308382 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:20.309131 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:20.309647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:20.310164 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:20.394460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:20.403105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-74158948-24b7-452d-bcb3-1666aa400b99 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.154333 Oct 28 11:12:20.403767 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 469/950] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:20 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:20.443595 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:20.451839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ce55d1df-efdc-4ca1-bdbb-41b515268d58 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1579 microversion: 2.1 time: 0.155235 Oct 28 11:12:20.452393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 482/951] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:20 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1579 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:12:21.426491 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:21.427023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:21.433828 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:21.434341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:21.434772 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:21.435456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:21.435906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:21.436325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:21.475536 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:21.476248 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:21.482751 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:21.483224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:21.483649 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:21.484332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:21.484806 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:21.485257 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:21.575583 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:21.584011 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5aaad5f1-767e-4bc5-9609-1c778039e396 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.161496 Oct 28 11:12:21.584963 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 470/952] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:21 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:21.836646 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:21.845366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e4a9dd80-81b8-449c-b8f2-7e5611f4d248 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1579 microversion: 2.1 time: 0.373782 Oct 28 11:12:21.845945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 483/953] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:21 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1579 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:12:22.123686 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-668165da-d097-4124-9cc8-5e4423734268 req-e53b8fa7-d762-4633-9835-d49807f9c7c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a", "name": "network-vif-deleted", "tag": "77a42644-3142-4fb2-9d0e-5deb0792310a"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:22.134051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-668165da-d097-4124-9cc8-5e4423734268 req-e53b8fa7-d762-4633-9835-d49807f9c7c4 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.134526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-668165da-d097-4124-9cc8-5e4423734268 req-e53b8fa7-d762-4633-9835-d49807f9c7c4 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.135139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-668165da-d097-4124-9cc8-5e4423734268 req-e53b8fa7-d762-4633-9835-d49807f9c7c4 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.147245 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-668165da-d097-4124-9cc8-5e4423734268 req-e53b8fa7-d762-4633-9835-d49807f9c7c4 service nova] Creating event network-vif-deleted:77a42644-3142-4fb2-9d0e-5deb0792310a for instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a on hv-cinder-zed-3 Oct 28 11:12:22.152291 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-668165da-d097-4124-9cc8-5e4423734268 req-e53b8fa7-d762-4633-9835-d49807f9c7c4 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032761 Oct 28 11:12:22.153202 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 471/954] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:12:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:12:22.608387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:22.608985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:22.615110 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.615580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.616126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.616951 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.617393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.618004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.747588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:22.755444 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3e237d76-67d5-4097-a8d8-9cbca6925a83 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1508 microversion: 2.1 time: 0.151411 Oct 28 11:12:22.755993 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 484/955] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:22 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1508 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:22.778838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:22.779527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:22.785714 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.786234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.786703 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.787397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.787851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.788277 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.868652 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:22.869257 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:22.873944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120544) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 28 11:12:22.875694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.876166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.876596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.877267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.877715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.878161 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.918540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.919004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.919625 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.920075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lazy-loading 'fault' on Instance uuid 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:12:22.950304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f71bcc17-4885-492d-b08c-829735e1d5db tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "POST /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735/action" status: 202 len: 0 microversion: 2.1 time: 0.176151 Oct 28 11:12:22.953901 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 472/956] 10.222.0.46 () {60 vars in 1278 bytes} [Fri Oct 28 11:12:22 2022] POST /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735/action => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:12:22.965447 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:22.966169 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:22.969073 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6b926f7161c0487a8c2b04ada3b374e3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9aff70e0d61c41fbb469ed4bcd785b23', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:22.976037 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.976524 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.976953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.977951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:22.978425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:22.978687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fc17988d-4b15-41cd-bd63-e7d72c88294f tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 200 len: 1231 microversion: 2.1 time: 0.113641 Oct 28 11:12:22.978969 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:22.980489 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 485/957] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:22 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 1231 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:12:23.122213 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:23.130614 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ceb95db7-4f1e-4c56-b3e7-360b7136fdbd tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1516 microversion: 2.1 time: 0.170724 Oct 28 11:12:23.131186 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 473/958] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:22 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1516 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:24.006235 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:24.006812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:24.013449 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.013912 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.014431 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.015123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.015581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.015997 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.030124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] HTTP exception thrown: Instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a could not be found. Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Request method failure captured: Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a HTTP/1.1 Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7bpKhsLly_AsLC5I5Ufi7RRtWwekiZ20oTVh9O4umzrCT6TaV8PH5o_R9SYBAOVP2EofrgrO2OiTcdq11VWJ9GiBna5aI-gXnS1_CYhIzgxrlZivt_OgETCTDDtqo0G4RtPvy7n959ltOqSgWxONV0J0lU0ZrvO6vH8e0oBtYQ2BtI Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:12:24.031778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 6b926f7161c0487a8c2b04ada3b374e3 Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a could not be found. Oct 28 11:12:24.033394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a'} Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a could not be found. Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:12:24.034308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a could not be found. Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a could not be found. Oct 28 11:12:24.035778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.037307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Returning 404 to user: Instance 7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:12:24.037307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-02697883-68bc-4fe3-aa05-d11ea971c5a4 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a" status: 404 len: 111 microversion: 2.1 time: 0.030857 Oct 28 11:12:24.037307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 486/959] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:24 2022] GET /compute/v2.1/servers/7528c29b-9ff1-47bc-b004-ab5c1c1bbd9a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:12:24.154352 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:24.155117 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:24.162058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.162556 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.162994 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.163699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.164155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.164582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.377582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:24.385604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-01b08cc2-c136-4938-85f1-c1ab13957670 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1518 microversion: 2.1 time: 0.235724 Oct 28 11:12:24.386764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 474/960] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:24 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1518 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:24.557164 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:24.557653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:24.564432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.564904 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.565330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.566028 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.566476 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.566892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.580817 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] HTTP exception thrown: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Request method failure captured: Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 HTTP/1.1 Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7bpKhsLly_AsLC5I5Ufi7RRtWwekiZ20oTVh9O4umzrCT6TaV8PH5o_R9SYBAOVP2EofrgrO2OiTcdq11VWJ9GiBna5aI-gXnS1_CYhIzgxrlZivt_OgETCTDDtqo0G4RtPvy7n959ltOqSgWxONV0J0lU0ZrvO6vH8e0oBtYQ2BtI Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:12:24.582342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 6b926f7161c0487a8c2b04ada3b374e3 Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.583949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '917e11e6-71de-4156-bffc-7a5a81135795'} Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:12:24.584802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.586058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.587325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Returning 404 to user: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:12:24.587325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-64bdad72-b60a-42c9-b05d-566430546377 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "DELETE /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 404 len: 111 microversion: 2.1 time: 0.030360 Oct 28 11:12:24.587325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 487/961] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:12:24 2022] DELETE /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:12:24.592495 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:24.593122 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] [instance: 917e11e6-71de-4156-bffc-7a5a81135795] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:24.599953 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.600426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.600858 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.601523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:24.601966 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:24.602405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:24.616791 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] HTTP exception thrown: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Request method failure captured: Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 HTTP/1.1 Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7bpKhsLly_AsLC5I5Ufi7RRtWwekiZ20oTVh9O4umzrCT6TaV8PH5o_R9SYBAOVP2EofrgrO2OiTcdq11VWJ9GiBna5aI-gXnS1_CYhIzgxrlZivt_OgETCTDDtqo0G4RtPvy7n959ltOqSgWxONV0J0lU0ZrvO6vH8e0oBtYQ2BtI Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:12:24.618448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9aff70e0d61c41fbb469ed4bcd785b23"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 9aff70e0d61c41fbb469ed4bcd785b23 Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-TestVolumeBootPattern-1555931021 Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: 6b926f7161c0487a8c2b04ada3b374e3 Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-TestVolumeBootPattern-1555931021-project Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.619784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '917e11e6-71de-4156-bffc-7a5a81135795'} Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:12:24.621088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. Oct 28 11:12:24.622746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:12:24.624408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Returning 404 to user: Instance 917e11e6-71de-4156-bffc-7a5a81135795 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:12:24.624408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-54092baa-fede-48d2-b065-6984f4312a32 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795" status: 404 len: 111 microversion: 2.1 time: 0.031117 Oct 28 11:12:24.624408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 475/962] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:24 2022] GET /compute/v2.1/servers/917e11e6-71de-4156-bffc-7a5a81135795 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 11:12:25.415498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-07972bcc-d8b9-4d70-a411-da340a8de3a0 req-5f20d17e-788a-4c3c-bf88-b6dbbe5f0e20 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c0936af-77e0-4b5c-b84c-08f6f5d72735", "name": "network-vif-unplugged", "status": "completed", "tag": "0404e8da-6840-4714-b253-a87262494023"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:25.420968 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:25.421711 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:25.428494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:25.428972 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:25.429398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:25.430116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:25.430563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:25.430983 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:25.439690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-07972bcc-d8b9-4d70-a411-da340a8de3a0 req-5f20d17e-788a-4c3c-bf88-b6dbbe5f0e20 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:25.440201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-07972bcc-d8b9-4d70-a411-da340a8de3a0 req-5f20d17e-788a-4c3c-bf88-b6dbbe5f0e20 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:25.440572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-07972bcc-d8b9-4d70-a411-da340a8de3a0 req-5f20d17e-788a-4c3c-bf88-b6dbbe5f0e20 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:25.452273 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-07972bcc-d8b9-4d70-a411-da340a8de3a0 req-5f20d17e-788a-4c3c-bf88-b6dbbe5f0e20 service nova] Creating event network-vif-unplugged:0404e8da-6840-4714-b253-a87262494023 for instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 on hv-cinder-zed-3 Oct 28 11:12:25.457498 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-07972bcc-d8b9-4d70-a411-da340a8de3a0 req-5f20d17e-788a-4c3c-bf88-b6dbbe5f0e20 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.110187 Oct 28 11:12:25.458112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 488/963] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:12:25 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:12:25.563217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:25.570778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-efcc1d09-7ddf-404f-af5b-b59131fc0773 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.164768 Oct 28 11:12:25.571330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 476/964] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:25 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:12:25.917779 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-99bdf66f-4e5d-4c65-8003-84ce694fea59 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] Calling method '.version_select at 0x7f42fa85fca0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:25.924622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:12:25.938318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-99bdf66f-4e5d-4c65-8003-84ce694fea59 tempest-TestVolumeBootPattern-1555931021 tempest-TestVolumeBootPattern-1555931021-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1118103513" status: 202 len: 0 microversion: 2.1 time: 0.024891 Oct 28 11:12:25.938919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 489/965] 10.222.0.46 () {58 vars in 1263 bytes} [Fri Oct 28 11:12:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1118103513 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:12:26.594396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:26.595007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:26.601238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:26.601913 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:26.602370 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:26.603053 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:26.603499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:26.603914 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:26.737381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:26.745798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e2317532-b3b7-4b73-9a06-6d475b10ef80 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.154759 Oct 28 11:12:26.745798 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 477/966] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:26 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:27.768266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:27.769582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:27.775960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:27.777253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:27.777847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:27.778714 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:27.779320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:27.779893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:27.922071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:27.929939 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ea2fd2aa-e28f-4ff5-879a-64b272160732 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.165714 Oct 28 11:12:27.930568 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 490/967] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:27 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:28.953399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:28.954043 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:28.957830 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=64,nova_api:UPDATE=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:12:28.964194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:28.964970 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:28.965500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:28.966416 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:28.967034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:28.967624 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:29.186875 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:29.195710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3a8dfc32-eac7-4b5a-8b3e-bab83d44eeb5 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.246661 Oct 28 11:12:29.196506 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 478/968] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:28 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:30.219924 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:30.220555 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:30.226522 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:30.226996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:30.227424 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:30.228101 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:30.228546 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:30.228967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:30.466850 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:30.474608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-55ac1bbb-1f27-4c2f-b7d0-bb1bdc29113e tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.258487 Oct 28 11:12:30.475112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 491/969] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:30 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:31.503370 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:31.503989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:31.510408 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:31.511455 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:31.511883 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:31.512573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:31.513024 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:31.513449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:31.649380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:31.657819 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-271a32f1-8891-43bd-b589-326055c3d7d6 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.163585 Oct 28 11:12:31.658500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 479/970] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:31 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:32.680837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:32.681549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:32.687683 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:32.688166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:32.688778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:32.689477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:32.689933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:32.690793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:32.827657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:32.835287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-341e5532-b01f-4ee7-8e60-2db30c50bfa8 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.158494 Oct 28 11:12:32.836042 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 492/971] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:32 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:33.859460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:33.860072 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:33.866972 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:33.867479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:33.867913 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:33.868603 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:33.869060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:33.869487 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:34.002754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:34.013368 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-31bb4527-be95-409d-85aa-53653a70d31d tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1574 microversion: 2.1 time: 0.157710 Oct 28 11:12:34.013964 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 480/972] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:33 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1574 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:35.036364 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:35.037072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:35.042956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:35.043442 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:35.043879 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:35.044566 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:35.045027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:35.045450 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:35.178071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:12:35.186452 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-42c2d426-94f1-4c84-af42-e6ac1de3cfbb tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1561 microversion: 2.1 time: 0.153892 Oct 28 11:12:35.187020 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 493/973] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:12:35 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1561 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:12:35.913999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-e0029590-702d-4395-99f4-b4a55c2f88c2 req-4456259d-e54d-459e-b8ab-963831a8f4b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c0936af-77e0-4b5c-b84c-08f6f5d72735", "name": "network-vif-plugged", "status": "completed", "tag": "0404e8da-6840-4714-b253-a87262494023"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:12:35.926364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e0029590-702d-4395-99f4-b4a55c2f88c2 req-4456259d-e54d-459e-b8ab-963831a8f4b8 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:35.927100 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e0029590-702d-4395-99f4-b4a55c2f88c2 req-4456259d-e54d-459e-b8ab-963831a8f4b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:35.927526 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-e0029590-702d-4395-99f4-b4a55c2f88c2 req-4456259d-e54d-459e-b8ab-963831a8f4b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:35.939331 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-e0029590-702d-4395-99f4-b4a55c2f88c2 req-4456259d-e54d-459e-b8ab-963831a8f4b8 service nova] Creating event network-vif-plugged:0404e8da-6840-4714-b253-a87262494023 for instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 on hv-cinder-zed-3 Oct 28 11:12:35.944177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-e0029590-702d-4395-99f4-b4a55c2f88c2 req-4456259d-e54d-459e-b8ab-963831a8f4b8 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034331 Oct 28 11:12:35.944995 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 481/974] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:12:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:12:59.673794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:12:59.674396 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:12:59.676351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=10,nova_api:DELETE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:12:59.677013 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:12:59.682902 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:59.683385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:59.683821 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:59.685023 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:12:59.685485 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:12:59.685910 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:12:59.710741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-22405317-aca5-4513-ba63-8cf13aa58846 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/ips" status: 200 len: 149 microversion: 2.1 time: 0.041107 Oct 28 11:12:59.711306 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 494/975] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:12:59 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/ips => generated 149 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:01.059467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=7 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:01.060923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:01.065454 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "658354dc-fdd3-41a6-83d9-be16deaa0d8d"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:01.067849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:01.075522 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:01.075999 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:01.076424 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:01.077200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:01.077651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:01.078115 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:01.319143 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-08077b0a-e0c1-43f0-bfb2-434f7fb3671c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.260990 Oct 28 11:13:01.319912 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 482/976] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:13:01 2022] POST /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments => generated 194 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:08.856690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-15eb435d-5ec8-4edf-9b35-b5e9235ba0e6 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: .version_select at 0x7f42fa456280>, body: {"keypair": {"name": "tempest-keypair-623414710", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP08ou+teE+jqg1CpmObuPCVgaxq7YrvkcL93tTirJrc+4eYPzZQuss2/5G4QbzB6IXhgIm3lQe+P0s/VSeFocNLh3uW+0gTrorw5HF47u5V0Ly/Kz1LIwpfghOIseiH0g=="}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:08.872608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-15eb435d-5ec8-4edf-9b35-b5e9235ba0e6 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:13:08.877075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-15eb435d-5ec8-4edf-9b35-b5e9235ba0e6 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:13:08.898835 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-15eb435d-5ec8-4edf-9b35-b5e9235ba0e6 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.046041 Oct 28 11:13:08.899417 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 495/977] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:13:08 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:10.139258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-630304979", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "447754b6-a798-4b6b-beaa-8e9ec6ea9fd5"}], "security_groups": [{"name": "tempest-securitygroup--508245247"}], "key_name": "tempest-keypair-623414710", "user_data": "}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:10.239975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] validate_networks() for [('447754b6-a798-4b6b-beaa-8e9ec6ea9fd5', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:13:10.612238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:13:10.612670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:13:10.613133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:13:10.613529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:13:10.613930 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:13:10.622742 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:10.623218 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:10.623644 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:10.624381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:10.624835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:10.625251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:10.641692 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:13:10.645525 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:13:10.656116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:13:10.658449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:13:10.704575 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c67edb96-8c3e-44bc-890d-e359c493997b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.570416 Oct 28 11:13:10.705263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 483/978] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:13:10 2022] POST /compute/v2.1/servers => generated 401 bytes in 572 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:13:10.716524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dfcfa41c-c637-42ea-8be2-ee90fcb7e1d2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:10.717138 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dfcfa41c-c637-42ea-8be2-ee90fcb7e1d2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:10.719059 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:10.771731 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-dfcfa41c-c637-42ea-8be2-ee90fcb7e1d2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:10.779285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dfcfa41c-c637-42ea-8be2-ee90fcb7e1d2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1240 microversion: 2.1 time: 0.066731 Oct 28 11:13:10.779847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 496/979] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:10 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1240 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:11.803684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:11.804806 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:11.812097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:11.812590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:11.813033 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:11.813736 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:11.814409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:11.814963 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:11.891129 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:11.900513 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a43f3de8-6a58-4f4b-87dd-d60b7292ee26 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1310 microversion: 2.1 time: 0.101825 Oct 28 11:13:11.901083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 484/980] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:11 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1310 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:12.924858 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:12.925548 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:12.931536 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:12.932115 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:12.932618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:12.933408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:12.933926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:12.934956 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:13.068338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:13.076606 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5dc2657a-c319-490b-a679-b29e3d8865bc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.156445 Oct 28 11:13:13.077265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 497/981] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:12 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:13.140052 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-47ebd98b-7617-4acc-817c-a5e19f024640 req-cbd9d6d0-29c8-4778-a9aa-2b97d20bb739 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ddc0c4cb-9db0-435e-b0bf-64320ce1c24d", "tag": "3ddd409d-33e1-4877-aaa6-af35efa088ab"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:13.152962 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-47ebd98b-7617-4acc-817c-a5e19f024640 req-cbd9d6d0-29c8-4778-a9aa-2b97d20bb739 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:13.154055 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-47ebd98b-7617-4acc-817c-a5e19f024640 req-cbd9d6d0-29c8-4778-a9aa-2b97d20bb739 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:13.155111 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-47ebd98b-7617-4acc-817c-a5e19f024640 req-cbd9d6d0-29c8-4778-a9aa-2b97d20bb739 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:13.168072 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-47ebd98b-7617-4acc-817c-a5e19f024640 req-cbd9d6d0-29c8-4778-a9aa-2b97d20bb739 service nova] Creating event network-changed:3ddd409d-33e1-4877-aaa6-af35efa088ab for instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d on hv-cinder-zed-3 Oct 28 11:13:13.173029 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-47ebd98b-7617-4acc-817c-a5e19f024640 req-cbd9d6d0-29c8-4778-a9aa-2b97d20bb739 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037320 Oct 28 11:13:13.174359 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 485/982] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:13:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:13.727734 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:13.729873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:13.735733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:13.736206 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:13.736633 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:13.737313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:13.737762 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:13.738213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:13.766726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c5ac78c8-243e-46c1-a268-5f62736ab2ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.042861 Oct 28 11:13:13.767268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 498/983] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:13:13 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:13:13.833735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:13.834384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:13.840509 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:13.840982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:13.841410 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:13.842121 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:13.842569 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:13.842986 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:13.866950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:13:13.938829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b5ced76d-be58-4bd7-bce3-eb5eabf48976 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 204 len: 0 microversion: 2.1 time: 0.109754 Oct 28 11:13:13.939974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 486/984] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:13:13 2022] DELETE /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:13:13.948115 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:13.948657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:13.954447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:13.955527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:13.955964 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:13.956634 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:13.957079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:13.957493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:14.081401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:14.089449 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8a9dc138-8ad7-4b1a-9175-5b14d47e7e5f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1671 microversion: 2.1 time: 0.144995 Oct 28 11:13:14.090071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 499/985] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:13 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1671 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:14.101150 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:14.101851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:14.107712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:14.108190 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:14.108626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:14.109312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:14.109765 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:14.110226 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:14.235615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:14.243933 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6ac2bde3-97c1-429e-9d79-88b8030ae86d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.147120 Oct 28 11:13:14.244489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 487/986] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:14 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:15.266986 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:15.267666 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:15.269389 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:15.270115 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:15.274835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:15.275313 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:15.275740 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:15.276338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:15.276468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:15.276829 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:15.276954 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:15.277263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:15.277365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:15.277954 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:15.278447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:15.278873 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:15.407012 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:15.414064 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:15.416626 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f2a644cb-3979-46b7-b19c-c26f6f93fa76 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1671 microversion: 2.1 time: 0.307480 Oct 28 11:13:15.417203 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 500/987] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:15 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1671 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:15.421498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-34332101-048c-408d-9ec8-1c4ac1b86b63 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.158545 Oct 28 11:13:15.422062 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 488/988] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:15 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:16.440004 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:16.440608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:16.444253 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:16.444857 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:16.446539 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:16.447019 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:16.447455 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:16.448137 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:16.448583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:16.448997 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:16.450850 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:16.451322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:16.451747 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:16.452424 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:16.452871 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:16.453284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:16.579109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:16.585941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:16.588032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-680979ba-d660-47ef-a9d5-d9f8bd7de309 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.147752 Oct 28 11:13:16.588712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 489/989] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:16 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:16.601627 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8e96caf4-fbe5-4999-9309-cd8439dc21a3 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1671 microversion: 2.1 time: 0.165380 Oct 28 11:13:16.602435 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 501/990] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:16 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1671 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:17.612032 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:17.612665 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:17.619846 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:17.620368 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:17.620800 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:17.621599 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:17.622082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:17.622506 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:17.624588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:17.625349 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:17.632223 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:17.632706 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:17.633144 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:17.633834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:17.634328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:17.634755 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:17.804503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:17.812749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4a80343f-fdb2-4a94-83d3-1a30cf5dbbb8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1671 microversion: 2.1 time: 0.192029 Oct 28 11:13:17.828024 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 502/991] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:17 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1671 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:17.828367 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:17.828367 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1fbd9b5b-d816-4519-8ba6-fe6ecb100360 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.218537 Oct 28 11:13:17.838923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 490/992] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:17 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:18.582633 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-c6ee0b3d-9ff6-404d-8aad-479c6f78c906 req-e55644bd-2cf4-4900-87bb-a454788595bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4432177-31d4-43c8-a6a2-a1d727b144a2", "name": "network-vif-deleted", "tag": "c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:18.598766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c6ee0b3d-9ff6-404d-8aad-479c6f78c906 req-e55644bd-2cf4-4900-87bb-a454788595bc service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:18.599502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c6ee0b3d-9ff6-404d-8aad-479c6f78c906 req-e55644bd-2cf4-4900-87bb-a454788595bc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:18.600093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-c6ee0b3d-9ff6-404d-8aad-479c6f78c906 req-e55644bd-2cf4-4900-87bb-a454788595bc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:18.612793 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-c6ee0b3d-9ff6-404d-8aad-479c6f78c906 req-e55644bd-2cf4-4900-87bb-a454788595bc service nova] Creating event network-vif-deleted:c95f96b2-ea44-4aa5-aeb1-ebbd21ec24e0 for instance e4432177-31d4-43c8-a6a2-a1d727b144a2 on hv-cinder-zed-3 Oct 28 11:13:18.618612 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-c6ee0b3d-9ff6-404d-8aad-479c6f78c906 req-e55644bd-2cf4-4900-87bb-a454788595bc service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040243 Oct 28 11:13:18.619292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 503/993] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:13:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:18.838597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:18.839198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:18.847232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:18.847716 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:18.848356 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:18.849327 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:18.850097 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:18.850673 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:18.852399 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:18.853106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:18.859288 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:18.859777 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:18.860216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:18.860920 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:18.861661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:18.862358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:18.886185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:18.886677 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:18.887102 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:18.887775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lazy-loading 'fault' on Instance uuid e4432177-31d4-43c8-a6a2-a1d727b144a2 {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:13:18.931226 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:18.940086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c8469e6a-5fb9-4edb-bbb3-786dbe35d81b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 200 len: 1327 microversion: 2.1 time: 0.104922 Oct 28 11:13:18.941005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 491/994] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:18 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 1327 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:19.005859 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:19.013816 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-50fa488d-56f4-4f16-8ff9-07517dc65346 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.165600 Oct 28 11:13:19.014424 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 504/995] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:18 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:19.963980 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:19.964547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:19.971464 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:19.971941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:19.972370 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:19.973056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:19.973502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:19.973919 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:19.987586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] HTTP exception thrown: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Request method failure captured: Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 HTTP/1.1 Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7gfPevt3Jy22gRQNYvOjgge97akpdlKosCG4oPLI8otHxDouAaic-E4LXbphPWG70JOFkrtElTMcxuCNeP2S-mTWhhY1brMe1RcH188jj9YBxcN_PwE3wblubAQuB8v_vUTtjBSkVxncszTFpB8qQqk4G9xnq65oTjtP_Vpe4hObnM Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:13:19.989215 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/64177183598843a68fc7fbe8257d4881"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_64177183598843a68fc7fbe8257d4881"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: a1f71865b51b4e8480932fffae985567 Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:13:19.991189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'e4432177-31d4-43c8-a6a2-a1d727b144a2'} Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:13:19.992118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:13:19.993566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:13:19.994782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Returning 404 to user: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:13:19.994782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-13e9262a-6115-4460-879d-195c55c19f59 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 404 len: 111 microversion: 2.1 time: 0.030569 Oct 28 11:13:19.994782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 492/996] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:19 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:13:20.006242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9852c187-7760-44ce-8979-80d1124b477f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '.version_select at 0x7f42fa85f1f0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:20.025486 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9852c187-7760-44ce-8979-80d1124b477f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-804514549" status: 202 len: 0 microversion: 2.1 time: 0.023546 Oct 28 11:13:20.026104 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 505/997] 10.222.0.46 () {58 vars in 1218 bytes} [Fri Oct 28 11:13:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-804514549 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:13:20.042906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:20.043515 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:20.050160 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:20.050656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:20.051095 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:20.051804 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:20.052258 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:20.052678 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:20.134136 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-6a752fa6-be31-4551-9193-fd092bb06a14 req-5544d564-89d2-4d18-8e8b-0b0bfd8fc361 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ddc0c4cb-9db0-435e-b0bf-64320ce1c24d", "name": "network-vif-plugged", "status": "completed", "tag": "3ddd409d-33e1-4877-aaa6-af35efa088ab"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:20.145365 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6a752fa6-be31-4551-9193-fd092bb06a14 req-5544d564-89d2-4d18-8e8b-0b0bfd8fc361 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:20.145967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6a752fa6-be31-4551-9193-fd092bb06a14 req-5544d564-89d2-4d18-8e8b-0b0bfd8fc361 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:20.146556 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6a752fa6-be31-4551-9193-fd092bb06a14 req-5544d564-89d2-4d18-8e8b-0b0bfd8fc361 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:20.157784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-6a752fa6-be31-4551-9193-fd092bb06a14 req-5544d564-89d2-4d18-8e8b-0b0bfd8fc361 service nova] Creating event network-vif-plugged:3ddd409d-33e1-4877-aaa6-af35efa088ab for instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d on hv-cinder-zed-3 Oct 28 11:13:20.163003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-6a752fa6-be31-4551-9193-fd092bb06a14 req-5544d564-89d2-4d18-8e8b-0b0bfd8fc361 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033437 Oct 28 11:13:20.164165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 506/998] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:13:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:13:20.241373 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:20.249457 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-aa8458f2-4627-4d8d-928c-f58786addf2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.210382 Oct 28 11:13:20.254059 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 493/999] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:20 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:20.745500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2bd6137e-f113-4f50-bef6-2bbb71aca1ed tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'create', calling method: .version_select at 0x7f42fa3f30d0>, body: {"keypair": {"name": "tempest-keypair-1622864765", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLR22+yrAdLRfsh93cgAC4h1dRJ3FohSmOUmGxN8tL2Ke+A/DhCRjCkloPxlbru8btbfi6qTUR4mR9pIzhKFZkmdkAEtn0PvnPv/F8SpjV8kPq/AuVhyV/iwZUnjWXwxmg=="}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:20.761252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-2bd6137e-f113-4f50-bef6-2bbb71aca1ed tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for project 64177183598843a68fc7fbe8257d4881. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:13:20.764992 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-2bd6137e-f113-4f50-bef6-2bbb71aca1ed tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for user a1f71865b51b4e8480932fffae985567 and project 64177183598843a68fc7fbe8257d4881. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:13:20.787701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2bd6137e-f113-4f50-bef6-2bbb71aca1ed tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.046463 Oct 28 11:13:20.788300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 507/1000] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:13:20 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:21.274688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:21.275406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:21.277399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:21.278486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:21.285202 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:21.285692 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:21.286162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:21.286867 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:21.287355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:21.287782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:21.419477 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:21.427740 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d033c09f-469d-4735-8156-cfce29a2d707 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1365 microversion: 2.1 time: 0.157231 Oct 28 11:13:21.428304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 494/1001] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:21 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:21.984319 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-102127101", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6b052aa7-a953-4b1a-862c-d4362c935620"}], "security_groups": [{"name": "tempest-securitygroup--1179679362"}], "key_name": "tempest-keypair-1622864765", "user_data": "}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:22.087538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] validate_networks() for [('6b052aa7-a953-4b1a-862c-d4362c935620', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:13:22.450998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:22.451740 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:22.458266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:22.458814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:22.459311 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:22.460065 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:22.460582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:22.461088 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:22.552888 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:13:22.553378 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:13:22.553921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:13:22.554419 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:13:22.554919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:13:22.563960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:22.565084 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:22.565530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:22.566366 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:22.566826 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:22.567255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:22.584015 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for project 64177183598843a68fc7fbe8257d4881. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:13:22.588185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Getting quotas for user a1f71865b51b4e8480932fffae985567 and project 64177183598843a68fc7fbe8257d4881. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:13:22.599635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:13:22.601988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:13:22.653266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-129b16dd-d8cf-45a8-b68f-3272490c4d00 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.673164 Oct 28 11:13:22.654356 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 508/1002] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:13:21 2022] POST /compute/v2.1/servers => generated 402 bytes in 675 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:13:22.665422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7fa84f58-ffa7-4e11-8cd3-248a475fa221 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:22.666285 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7fa84f58-ffa7-4e11-8cd3-248a475fa221 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:22.722803 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-7fa84f58-ffa7-4e11-8cd3-248a475fa221 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:22.731604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7fa84f58-ffa7-4e11-8cd3-248a475fa221 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1237 microversion: 2.1 time: 0.070032 Oct 28 11:13:22.732239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 509/1003] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:22 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1237 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:22.892922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:22.901167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3edb30b5-8694-4b65-875f-46029b7ec73a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1555 microversion: 2.1 time: 0.454596 Oct 28 11:13:22.901768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 495/1004] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:22 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1555 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:22.924655 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:22.925167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:22.931222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:22.931710 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:22.932142 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:22.932840 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:22.933292 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:22.933712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:23.017435 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f4524412-26f0-4463-9bba-c0b10d4cdd2d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.097076 Oct 28 11:13:23.018093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 510/1005] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:13:22 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d/os-interface => generated 285 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:23.754976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:23.755586 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:23.762814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:23.763300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:23.763735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:23.764439 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:23.764899 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:23.765322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:23.833685 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:23.842263 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ae03afce-c89b-4711-ab84-198099a6fab9 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1289 microversion: 2.1 time: 0.091150 Oct 28 11:13:23.842989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 496/1006] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:23 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1289 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:24.436298 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-d0f71092-482b-42ac-870e-158fb9fc6cdc req-da5ab470-11f2-4389-93c6-f1e377bb7267 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ddc0c4cb-9db0-435e-b0bf-64320ce1c24d", "tag": "3ddd409d-33e1-4877-aaa6-af35efa088ab"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:24.446701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d0f71092-482b-42ac-870e-158fb9fc6cdc req-da5ab470-11f2-4389-93c6-f1e377bb7267 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:24.447187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d0f71092-482b-42ac-870e-158fb9fc6cdc req-da5ab470-11f2-4389-93c6-f1e377bb7267 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:24.447620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-d0f71092-482b-42ac-870e-158fb9fc6cdc req-da5ab470-11f2-4389-93c6-f1e377bb7267 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:24.459738 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-d0f71092-482b-42ac-870e-158fb9fc6cdc req-da5ab470-11f2-4389-93c6-f1e377bb7267 service nova] Creating event network-changed:3ddd409d-33e1-4877-aaa6-af35efa088ab for instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d on hv-cinder-zed-3 Oct 28 11:13:24.464652 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-d0f71092-482b-42ac-870e-158fb9fc6cdc req-da5ab470-11f2-4389-93c6-f1e377bb7267 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032140 Oct 28 11:13:24.465250 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 511/1007] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:13:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:24.866641 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:24.869474 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:24.877522 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:24.882478 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:24.882925 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:24.883888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:24.884509 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:24.885132 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:24.912409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:25.193228 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:25.201039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-8bc9ae18-4800-43f2-a62b-42e6bc629d2d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.338511 Oct 28 11:13:25.201832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 497/1008] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:24 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:26.229618 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:26.230252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:26.236354 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:26.236832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:26.237300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:26.237950 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:26.238420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:26.238836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:26.366171 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:26.373733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9b1364a7-47d2-414a-9a5e-866ad701c30d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.148440 Oct 28 11:13:26.374320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 512/1009] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:26 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:26.479975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-c3d0c7bb-6653-4f67-bd8b-51c54b1da2aa req-369f97a5-05e1-40e5-832a-13b4bca54402 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db8a46ce-9e4d-4366-949b-b4584021c35b", "tag": "d701ecd4-e08b-4414-80c4-ca9da96dd6bc"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:26.489652 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-c3d0c7bb-6653-4f67-bd8b-51c54b1da2aa req-369f97a5-05e1-40e5-832a-13b4bca54402 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:26.490171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-c3d0c7bb-6653-4f67-bd8b-51c54b1da2aa req-369f97a5-05e1-40e5-832a-13b4bca54402 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:26.490863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-c3d0c7bb-6653-4f67-bd8b-51c54b1da2aa req-369f97a5-05e1-40e5-832a-13b4bca54402 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:26.501442 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-c3d0c7bb-6653-4f67-bd8b-51c54b1da2aa req-369f97a5-05e1-40e5-832a-13b4bca54402 service nova] Creating event network-changed:d701ecd4-e08b-4414-80c4-ca9da96dd6bc for instance db8a46ce-9e4d-4366-949b-b4584021c35b on hv-cinder-zed-3 Oct 28 11:13:26.506881 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-c3d0c7bb-6653-4f67-bd8b-51c54b1da2aa req-369f97a5-05e1-40e5-832a-13b4bca54402 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030734 Oct 28 11:13:26.507445 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 498/1010] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:13:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:27.396242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:27.396845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:27.402872 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:27.403348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:27.403777 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:27.404451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:27.404897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:27.405315 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:27.535223 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:27.543390 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-12b3853f-7a4a-402e-9a4e-afb75480b1ad tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.151067 Oct 28 11:13:27.543954 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 513/1011] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:27 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:28.566849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:28.567544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:28.573688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:28.574194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:28.575187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:28.575343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:28.575796 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:28.576212 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:28.701529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:28.709712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-990d88c3-9c75-4ff3-ae03-9f2f66bd5224 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.147641 Oct 28 11:13:28.710304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 499/1012] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:28 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:29.733347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:29.733959 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:29.735972 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=5,nova_api:DELETE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:29.742116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:29.742588 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:29.743018 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:29.743693 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:29.744146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:29.744561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:29.870049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:29.877510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f5e0afdd-36a9-4b57-b5af-59661b335750 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.148036 Oct 28 11:13:29.878092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 514/1013] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:29 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:13:30.900319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:30.900927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:30.907316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:30.907797 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:30.908233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:30.908921 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:30.909384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:30.909809 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:31.038444 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:31.045958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-db1bc56c-953f-441c-98a6-b0f14f438d21 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.149787 Oct 28 11:13:31.046536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 500/1014] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:30 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:32.068838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:32.069466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:32.076997 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:32.077506 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:32.078120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:32.078987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:32.079442 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:32.079891 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:32.208189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-66a7ac17-e431-40fe-9026-a5178cc9578e req-571b0d5f-6edc-46e1-90ee-c8a2d004bcd9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db8a46ce-9e4d-4366-949b-b4584021c35b", "name": "network-vif-plugged", "status": "completed", "tag": "d701ecd4-e08b-4414-80c4-ca9da96dd6bc"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:32.215550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:32.217831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:32.219868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-66a7ac17-e431-40fe-9026-a5178cc9578e req-571b0d5f-6edc-46e1-90ee-c8a2d004bcd9 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:32.220426 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-66a7ac17-e431-40fe-9026-a5178cc9578e req-571b0d5f-6edc-46e1-90ee-c8a2d004bcd9 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:32.220901 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-66a7ac17-e431-40fe-9026-a5178cc9578e req-571b0d5f-6edc-46e1-90ee-c8a2d004bcd9 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:32.224065 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-03f1eb54-217e-4a4b-900e-30b3f30c90d5 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.159271 Oct 28 11:13:32.224878 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 515/1015] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:32 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:32.235314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-66a7ac17-e431-40fe-9026-a5178cc9578e req-571b0d5f-6edc-46e1-90ee-c8a2d004bcd9 service nova] Creating event network-vif-plugged:d701ecd4-e08b-4414-80c4-ca9da96dd6bc for instance db8a46ce-9e4d-4366-949b-b4584021c35b on hv-cinder-zed-3 Oct 28 11:13:32.241548 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-66a7ac17-e431-40fe-9026-a5178cc9578e req-571b0d5f-6edc-46e1-90ee-c8a2d004bcd9 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036700 Oct 28 11:13:32.242175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 501/1016] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:13:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:13:33.248014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:33.248553 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:33.250405 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:33.251263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=18 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:13:33.257511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:33.257995 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:33.258449 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:33.259140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:33.259592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:33.260015 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:33.385466 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:33.393076 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c928dcca-031b-47d1-9faa-dfea417a18da tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1363 microversion: 2.1 time: 0.149052 Oct 28 11:13:33.393639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 516/1017] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:33 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1363 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:34.415860 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:34.416660 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:34.424287 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:34.424760 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:34.425188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:34.425902 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:34.426882 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:34.427300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:34.560670 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:13:34.568855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-33bd3798-6f62-4065-bc53-7440a78537b8 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1548 microversion: 2.1 time: 0.157241 Oct 28 11:13:34.569407 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 502/1018] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:13:34 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1548 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:13:34.590733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:13:34.591247 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:13:34.597068 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:34.597545 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:34.598525 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:34.599213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:34.599672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:34.600093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:34.691514 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-16ee1f3e-8b3f-477c-a91d-edca0aaadd22 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.104535 Oct 28 11:13:34.692425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 517/1019] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:13:34 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-interface => generated 284 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:13:35.972952 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-f2da8b04-bf0c-4ad0-ac5e-197c44c77cc2 req-6107b555-5f20-456a-9b13-3160b2ea562d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "db8a46ce-9e4d-4366-949b-b4584021c35b", "tag": "d701ecd4-e08b-4414-80c4-ca9da96dd6bc"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:13:35.983896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-f2da8b04-bf0c-4ad0-ac5e-197c44c77cc2 req-6107b555-5f20-456a-9b13-3160b2ea562d service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:13:35.984382 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-f2da8b04-bf0c-4ad0-ac5e-197c44c77cc2 req-6107b555-5f20-456a-9b13-3160b2ea562d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:13:35.984807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-f2da8b04-bf0c-4ad0-ac5e-197c44c77cc2 req-6107b555-5f20-456a-9b13-3160b2ea562d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:13:35.996763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-f2da8b04-bf0c-4ad0-ac5e-197c44c77cc2 req-6107b555-5f20-456a-9b13-3160b2ea562d service nova] Creating event network-changed:d701ecd4-e08b-4414-80c4-ca9da96dd6bc for instance db8a46ce-9e4d-4366-949b-b4584021c35b on hv-cinder-zed-3 Oct 28 11:13:36.001905 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-f2da8b04-bf0c-4ad0-ac5e-197c44c77cc2 req-6107b555-5f20-456a-9b13-3160b2ea562d service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033407 Oct 28 11:13:36.002547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 503/1020] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:13:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:19.573404 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:19.574029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:19.576111 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-64b4d394-cae0-41a4-866e-a14fe281e4eb req-12b4ae00-2930-4e20-b0a2-baca98dfddb8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c0936af-77e0-4b5c-b84c-08f6f5d72735", "tag": "0404e8da-6840-4714-b253-a87262494023"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:19.579308 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=7 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:19.579976 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=88 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:19.582199 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:19.582569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:19.582948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=5 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:19.583079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:19.583715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=28 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:19.583839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:19.585050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:19.585430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:19.591834 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-64b4d394-cae0-41a4-866e-a14fe281e4eb req-12b4ae00-2930-4e20-b0a2-baca98dfddb8 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:19.591834 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-64b4d394-cae0-41a4-866e-a14fe281e4eb req-12b4ae00-2930-4e20-b0a2-baca98dfddb8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:19.591834 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-64b4d394-cae0-41a4-866e-a14fe281e4eb req-12b4ae00-2930-4e20-b0a2-baca98dfddb8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:19.604236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-64b4d394-cae0-41a4-866e-a14fe281e4eb req-12b4ae00-2930-4e20-b0a2-baca98dfddb8 service nova] Creating event network-changed:0404e8da-6840-4714-b253-a87262494023 for instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 on hv-cinder-zed-3 Oct 28 11:14:19.609364 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-64b4d394-cae0-41a4-866e-a14fe281e4eb req-12b4ae00-2930-4e20-b0a2-baca98dfddb8 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037441 Oct 28 11:14:19.610417 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 504/1021] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:14:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:14:19.612948 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:14:19.698560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-68611275-6d32-45f5-a754-eac492866130 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "DELETE /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 204 len: 0 microversion: 2.1 time: 0.129418 Oct 28 11:14:19.702845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 518/1022] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:14:19 2022] DELETE /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:14:19.710498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:19.711256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:19.718074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:19.718559 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:19.718992 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:19.719688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:19.720141 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:19.720563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:19.873607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:19.882106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-bd9a1f79-8d9e-4782-9e08-744ed8b3b3cc tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.177083 Oct 28 11:14:19.882676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 505/1023] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:19 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:20.906222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:20.907014 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:20.913563 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:20.914070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:20.914505 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:20.915197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:20.915648 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:20.916070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:21.046521 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:21.054387 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ac85389b-cf4c-4627-a8aa-5802e960fe65 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1567 microversion: 2.1 time: 0.152445 Oct 28 11:14:21.054960 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 519/1024] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:20 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:21.860251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-406058e3-11be-418d-8242-fb505274c694 req-db4c8721-b177-4669-9817-63e243e51c20 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c0936af-77e0-4b5c-b84c-08f6f5d72735", "name": "network-vif-deleted", "tag": "0404e8da-6840-4714-b253-a87262494023"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:21.871557 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-406058e3-11be-418d-8242-fb505274c694 req-db4c8721-b177-4669-9817-63e243e51c20 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:21.872031 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-406058e3-11be-418d-8242-fb505274c694 req-db4c8721-b177-4669-9817-63e243e51c20 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:21.872460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-406058e3-11be-418d-8242-fb505274c694 req-db4c8721-b177-4669-9817-63e243e51c20 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:21.883746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-406058e3-11be-418d-8242-fb505274c694 req-db4c8721-b177-4669-9817-63e243e51c20 service nova] Creating event network-vif-deleted:0404e8da-6840-4714-b253-a87262494023 for instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 on hv-cinder-zed-3 Oct 28 11:14:21.888543 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-406058e3-11be-418d-8242-fb505274c694 req-db4c8721-b177-4669-9817-63e243e51c20 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031971 Oct 28 11:14:21.889137 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 506/1025] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:14:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:22.077858 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:22.078604 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:22.085635 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:22.086167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:22.086603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:22.087296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:22.087749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:22.088170 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:22.152919 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4ade5932a324a4b8956b5377256ce7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edc5ff928624d87944ee071e578aa17', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:22.161223 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-489ed24e-6d8f-4f47-8d25-660bfd0d552f tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 200 len: 1220 microversion: 2.1 time: 0.087045 Oct 28 11:14:22.162166 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 520/1026] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:22 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:23.185791 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:23.186462 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] [instance: 8c0936af-77e0-4b5c-b84c-08f6f5d72735] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:23.192645 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:23.193128 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:23.193559 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:23.194273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:23.194727 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:23.195148 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:23.208052 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] HTTP exception thrown: Instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 could not be found. Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Request method failure captured: Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 HTTP/1.1 Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7g3ONExJPRV1b1yuNQmEntB_c_xx70pTxtihKjtokQZDw1kKDe9DkxSZHlR9s8oxDg1c85rnKzVHJkOIn2cH7qRrezS--nHOonR9lZdtuBIUUxjU5X8YqbzaKY2F_DqMHUx3CSOY5SBVeJFv94G7EvnOeKBxHRLS8DukLNKEvP9nPI Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 9edc5ff928624d87944ee071e578aa17 Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-TestShelveInstance-1768554577 Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:14:23.210164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/9edc5ff928624d87944ee071e578aa17"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_9edc5ff928624d87944ee071e578aa17"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9edc5ff928624d87944ee071e578aa17"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/9edc5ff928624d87944ee071e578aa17"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-TestShelveInstance-1768554577 Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 9edc5ff928624d87944ee071e578aa17 Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-TestShelveInstance-1768554577 Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-TestShelveInstance-1768554577-project Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: d4ade5932a324a4b8956b5377256ce7a Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-TestShelveInstance-1768554577-project Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 could not be found. Oct 28 11:14:23.211936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '8c0936af-77e0-4b5c-b84c-08f6f5d72735'} Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 could not be found. Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:14:23.213101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 could not be found. Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 could not be found. Oct 28 11:14:23.214775 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:23.216325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Returning 404 to user: Instance 8c0936af-77e0-4b5c-b84c-08f6f5d72735 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:14:23.216325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-554a9ef2-5729-4bd2-b77d-d02cf80e951b tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735" status: 404 len: 111 microversion: 2.1 time: 0.030513 Oct 28 11:14:23.216325 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 507/1027] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:23 2022] GET /compute/v2.1/servers/8c0936af-77e0-4b5c-b84c-08f6f5d72735 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:14:23.368421 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a6a1bf43-ef5d-46fe-b347-fb5d4a495051 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] Calling method '.version_select at 0x7f42fa4d4280>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:23.387425 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a6a1bf43-ef5d-46fe-b347-fb5d4a495051 tempest-TestShelveInstance-1768554577 tempest-TestShelveInstance-1768554577-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1437936952" status: 202 len: 0 microversion: 2.1 time: 0.023228 Oct 28 11:14:23.387996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 521/1028] 10.222.0.46 () {58 vars in 1254 bytes} [Fri Oct 28 11:14:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1437936952 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:14:39.466034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:39.466590 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:39.468679 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=8 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:39.469324 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=12 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:39.475371 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.475869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.476302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.476985 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.477442 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.477859 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.501144 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-15fe3bfe-7e0a-4807-80c0-80abd6287ab0 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d/ips" status: 200 len: 149 microversion: 2.1 time: 0.039398 Oct 28 11:14:39.501695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 508/1029] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:14:39 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d/ips => generated 149 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:39.510721 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:39.511459 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:39.515459 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "658354dc-fdd3-41a6-83d9-be16deaa0d8d"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:39.517797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:39.525569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.526075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.526508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.527184 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.527628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.528043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.607400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Invalid volume: volume 658354dc-fdd3-41a6-83d9-be16deaa0d8d is already attached to instances: 952a161f-410d-4266-b236-14823a631423 Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: POST /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d/os-volume_attachments HTTP/1.1 Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 74 Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:14:39.609628 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {"volumeAttachment": {"volumeId": "658354dc-fdd3-41a6-83d9-be16deaa0d8d"}} Oct 28 11:14:39.611230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Invalid volume: volume 658354dc-fdd3-41a6-83d9-be16deaa0d8d is already attached to instances: 952a161f-410d-4266-b236-14823a631423 Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'server_id': 'ddc0c4cb-9db0-435e-b0bf-64320ce1c24d', 'body': {'volumeAttachment': {'volumeId': '658354dc-fdd3-41a6-83d9-be16deaa0d8d'}}} Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 658354dc-fdd3-41a6-83d9-be16deaa0d8d is already attached to instances: 952a161f-410d-4266-b236-14823a631423 Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:39.612079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 658354dc-fdd3-41a6-83d9-be16deaa0d8d is already attached to instances: 952a161f-410d-4266-b236-14823a631423 Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:39.613614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 658354dc-fdd3-41a6-83d9-be16deaa0d8d is already attached to instances: 952a161f-410d-4266-b236-14823a631423 Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 400 to user: Invalid volume: volume 658354dc-fdd3-41a6-83d9-be16deaa0d8d is already attached to instances: 952a161f-410d-4266-b236-14823a631423 {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5d8d9e1b-f762-41b6-87bc-172f886e29a7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.101326 Oct 28 11:14:39.615756 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 522/1030] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:14:39 2022] POST /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d/os-volume_attachments => generated 174 bytes in 102 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:14:39.620339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:39.620824 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:39.626638 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.627113 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.627543 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.628216 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.628660 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.629077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.653398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:14:39.725312 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-5e9935be-0d7d-4e47-9a38-b7636f846292 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 204 len: 0 microversion: 2.1 time: 0.108660 Oct 28 11:14:39.725887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 509/1031] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:14:39 2022] DELETE /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:14:39.735160 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:39.735799 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:39.743917 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.744397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.744825 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.745510 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:39.745958 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:39.746400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:39.877532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:39.885271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a6a34ae1-bdd2-4b87-a1ed-ca5df2681c6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1679 microversion: 2.1 time: 0.155070 Oct 28 11:14:39.885834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 523/1032] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:39 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1679 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:40.909929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:40.910646 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:40.917080 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:40.917588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:40.918031 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:40.918715 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:40.919162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:40.919578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:41.050613 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:41.058872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-76f094eb-c686-4531-a5c7-0b94ee5af8c2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1679 microversion: 2.1 time: 0.153298 Oct 28 11:14:41.059436 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 510/1033] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:40 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1679 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:42.040701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-8b52fb00-43e7-419e-bf18-310fe981c840 req-596a064f-92b4-4fc7-be38-b48d616892b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ddc0c4cb-9db0-435e-b0bf-64320ce1c24d", "name": "network-vif-deleted", "tag": "3ddd409d-33e1-4877-aaa6-af35efa088ab"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:42.050332 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8b52fb00-43e7-419e-bf18-310fe981c840 req-596a064f-92b4-4fc7-be38-b48d616892b8 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:42.050729 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8b52fb00-43e7-419e-bf18-310fe981c840 req-596a064f-92b4-4fc7-be38-b48d616892b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:42.051153 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-8b52fb00-43e7-419e-bf18-310fe981c840 req-596a064f-92b4-4fc7-be38-b48d616892b8 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:42.063312 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-8b52fb00-43e7-419e-bf18-310fe981c840 req-596a064f-92b4-4fc7-be38-b48d616892b8 service nova] Creating event network-vif-deleted:3ddd409d-33e1-4877-aaa6-af35efa088ab for instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d on hv-cinder-zed-3 Oct 28 11:14:42.069574 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-8b52fb00-43e7-419e-bf18-310fe981c840 req-596a064f-92b4-4fc7-be38-b48d616892b8 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033119 Oct 28 11:14:42.070327 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 524/1034] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:14:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:42.082560 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:42.083171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:42.098693 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:42.099233 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:42.099675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:42.100382 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:42.100835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:42.101264 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:42.195378 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:42.204940 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d3d9d521-71ca-4b8b-8de8-7fe557f644af tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 200 len: 1325 microversion: 2.1 time: 0.126208 Oct 28 11:14:42.205831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 511/1035] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:42 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 1325 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:43.229314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:43.230036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:43.236183 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:43.236665 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:43.237099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:43.237784 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:43.238264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:43.238692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:43.252047 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d HTTP/1.1 Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:14:43.253703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:14:43.255343 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'ddc0c4cb-9db0-435e-b0bf-64320ce1c24d'} Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:14:43.256260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:14:43.257743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:14:43.259230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:14:43.259230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-4f72b284-ce8a-4612-9adf-5baf762ca98c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 404 len: 111 microversion: 2.1 time: 0.030778 Oct 28 11:14:43.259230 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 525/1036] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:43 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:14:43.263969 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-82530a5a-1f54-45ca-994a-d08b049ff968 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '.version_select at 0x7f42fa65ab80>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:43.284351 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-82530a5a-1f54-45ca-994a-d08b049ff968 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-623414710" status: 202 len: 0 microversion: 2.1 time: 0.024128 Oct 28 11:14:43.284923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 512/1037] 10.222.0.46 () {58 vars in 1218 bytes} [Fri Oct 28 11:14:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-623414710 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:14:43.920049 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:43.920542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:43.926478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:43.926957 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:43.928328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:43.929227 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:43.929686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:43.930155 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:44.008353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lazy-loading 'services' on Instance uuid 952a161f-410d-4266-b236-14823a631423 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:14:44.077401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-70d897cb-e617-4bdd-8540-7de3d39974e9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments/658354dc-fdd3-41a6-83d9-be16deaa0d8d" status: 202 len: 0 microversion: 2.1 time: 0.161243 Oct 28 11:14:44.078038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 526/1038] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:14:43 2022] DELETE /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments/658354dc-fdd3-41a6-83d9-be16deaa0d8d => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:14:44.087046 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:44.089253 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:44.096301 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:44.096801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:44.097300 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:44.098176 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:44.098782 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:44.099860 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:44.129835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1df3e1fe-802e-4d24-aa33-3ac4c836c9ca tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047266 Oct 28 11:14:44.130467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 513/1039] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:14:44 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:45.143029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:45.145351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:45.152120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:45.152596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:45.153018 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:45.153690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:45.154164 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:45.154579 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:45.183245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-5eda39d2-6640-4694-ae74-f40112ea97b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044577 Oct 28 11:14:45.183794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 527/1040] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:14:45 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:14:45.245660 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:45.246204 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:45.252473 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:45.252950 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:45.253378 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:45.254084 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:45.254530 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:45.254948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:45.279922 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:14:45.354351 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2f11580e-6e6f-41ba-b913-703e54b63f28 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 204 len: 0 microversion: 2.1 time: 0.112446 Oct 28 11:14:45.354954 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 514/1041] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:14:45 2022] DELETE /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:14:45.363159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:45.363854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:45.370653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:45.371128 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:45.371557 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:45.372239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:45.372687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:45.373111 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:45.497532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:45.505141 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-270ff142-0a94-4922-8cb5-a663703e45f2 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1681 microversion: 2.1 time: 0.145787 Oct 28 11:14:45.505712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 528/1042] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:45 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1681 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:46.528280 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:46.528916 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:46.535313 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:46.535791 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:46.536222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:46.537409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:46.537861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:46.538309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:46.670810 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:46.679092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-99fb2a07-222b-4858-8e14-82eba2deacbc tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1681 microversion: 2.1 time: 0.155248 Oct 28 11:14:46.679662 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 515/1043] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:46 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:47.702582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:47.703196 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:47.710617 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:47.711102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:47.711538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:47.712229 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:47.712685 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:47.713105 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:47.780370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:47.788895 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f03307d8-d520-4096-957a-684ec4b838d3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 200 len: 1613 microversion: 2.1 time: 0.090177 Oct 28 11:14:47.789681 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 529/1044] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:47 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 1613 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:47.975214 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:47.975800 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:47.982136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:47.982688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:47.983128 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:47.983819 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:47.984269 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:47.984683 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:48.010116 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c1807d40-c2bb-48ee-8eed-6ce2535f656d tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/ips" status: 200 len: 144 microversion: 2.1 time: 0.039501 Oct 28 11:14:48.010878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 516/1045] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:14:47 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/ips => generated 144 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:48.048476 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-e1753146-3c6e-449c-abdc-464697051f89 req-d9bf898f-7608-4167-a6b8-34e1e9affff7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "952a161f-410d-4266-b236-14823a631423", "name": "network-vif-deleted", "tag": "da22476d-00ea-4b57-aedf-ea0bfc3eaaa2"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:48.058743 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e1753146-3c6e-449c-abdc-464697051f89 req-d9bf898f-7608-4167-a6b8-34e1e9affff7 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:48.059418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e1753146-3c6e-449c-abdc-464697051f89 req-d9bf898f-7608-4167-a6b8-34e1e9affff7 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:48.059899 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e1753146-3c6e-449c-abdc-464697051f89 req-d9bf898f-7608-4167-a6b8-34e1e9affff7 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:48.072794 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-e1753146-3c6e-449c-abdc-464697051f89 req-d9bf898f-7608-4167-a6b8-34e1e9affff7 service nova] Creating event network-vif-deleted:da22476d-00ea-4b57-aedf-ea0bfc3eaaa2 for instance 952a161f-410d-4266-b236-14823a631423 on hv-cinder-zed-3 Oct 28 11:14:48.078211 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-e1753146-3c6e-449c-abdc-464697051f89 req-d9bf898f-7608-4167-a6b8-34e1e9affff7 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033844 Oct 28 11:14:48.079517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 530/1046] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:14:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:48.812499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:48.813112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:48.819366 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:48.819851 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:48.820278 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:48.820965 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:48.821416 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:48.821833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:48.835924 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 HTTP/1.1 Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:14:48.837568 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:14:48.839334 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '952a161f-410d-4266-b236-14823a631423'} Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:14:48.840172 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:14:48.841460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:14:48.843600 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:14:48.843600 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ec686ad4-54da-4ad3-9577-ec5dc088bd79 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 404 len: 111 microversion: 2.1 time: 0.030388 Oct 28 11:14:48.843600 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 517/1047] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:48 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:14:48.848241 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7dc353b7-c77e-4168-8764-c359b8e6f85c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '.version_select at 0x7f42fa66b0d0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:48.869221 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7dc353b7-c77e-4168-8764-c359b8e6f85c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1930240948" status: 202 len: 0 microversion: 2.1 time: 0.025087 Oct 28 11:14:48.869812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 531/1048] 10.222.0.46 () {58 vars in 1221 bytes} [Fri Oct 28 11:14:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1930240948 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 11:14:49.456193 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3cb27c8c-dc5f-4f55-a688-ede7c1ae1227"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:49.459077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:49.465776 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:49.466310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:49.466758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:49.467467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:49.467923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:49.468347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:49.727073 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-097ca503-1201-48d0-bf65-71cd6a57c17f tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.274618 Oct 28 11:14:49.733008 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 518/1049] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:14:49 2022] POST /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments => generated 194 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:49.739642 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e5e49f62-5de2-4531-afea-dfc72ea879f9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: .version_select at 0x7f42fa4d44c0>, body: {"keypair": {"name": "tempest-keypair-43803389", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJzwBfwlIBBPpiQ1FV7W08wXqF4PcO7scukrvOKbZGySqBhHkLzQqbQnD+GMN235aWWBf7vgtQ0TaLikXJO0NNHYie+9XVeXGHWJkE3IDB4FzAuWWqmJKbwOz/3kJ79CJQ=="}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:49.756088 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-e5e49f62-5de2-4531-afea-dfc72ea879f9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:14:49.760150 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-e5e49f62-5de2-4531-afea-dfc72ea879f9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:14:49.784530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e5e49f62-5de2-4531-afea-dfc72ea879f9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.049400 Oct 28 11:14:49.790007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 532/1050] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:14:49 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:51.031177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-151739925", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "447754b6-a798-4b6b-beaa-8e9ec6ea9fd5"}], "security_groups": [{"name": "tempest-securitygroup--751662757"}], "key_name": "tempest-keypair-43803389", "user_data": "}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:51.138878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] validate_networks() for [('447754b6-a798-4b6b-beaa-8e9ec6ea9fd5', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:14:51.508693 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:14:51.509114 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:14:51.509582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:14:51.510000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:14:51.510448 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:14:51.519115 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:51.519597 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:51.520030 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:51.520777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:51.521232 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:51.521656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:51.537809 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:14:51.541693 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:14:51.552937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:14:51.555309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:14:51.602130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6e691d11-68ff-4280-88f7-c6b76dd339f4 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.575572 Oct 28 11:14:51.602864 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 519/1051] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:14:51 2022] POST /compute/v2.1/servers => generated 401 bytes in 577 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:14:51.613798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ca88bc72-54f3-4ccc-ada1-3c45a5a2c15d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:51.615087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ca88bc72-54f3-4ccc-ada1-3c45a5a2c15d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:51.670864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ca88bc72-54f3-4ccc-ada1-3c45a5a2c15d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:51.678658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ca88bc72-54f3-4ccc-ada1-3c45a5a2c15d tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1239 microversion: 2.1 time: 0.068808 Oct 28 11:14:51.690841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 533/1052] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:51 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1239 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:14:52.703070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:52.704740 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:52.712729 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:52.713528 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:52.714166 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:52.715070 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:52.715591 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:52.716026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:52.784488 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:52.792271 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-40086f03-5e7d-4d3f-822e-dc83ca09a3ec tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1297 microversion: 2.1 time: 0.094177 Oct 28 11:14:52.792833 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 520/1053] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:52 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1297 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:53.815650 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:53.816252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:53.822678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:53.823165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:53.823599 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:53.824322 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:53.824776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:53.825197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:53.953016 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:53.961338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8311ea1e-c7d6-48f1-8535-a297f0dc2813 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1364 microversion: 2.1 time: 0.149606 Oct 28 11:14:53.961914 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 534/1054] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:53 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1364 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:14:53.990820 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-a7436ace-5fa4-439c-a3ef-58eb041647e2 req-b159f001-71a3-49c3-b597-9b6ad6c9ef5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d", "tag": "8d461603-b871-45af-9a83-a23e1c61b1b6"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:54.004694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a7436ace-5fa4-439c-a3ef-58eb041647e2 req-b159f001-71a3-49c3-b597-9b6ad6c9ef5e service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:54.006774 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a7436ace-5fa4-439c-a3ef-58eb041647e2 req-b159f001-71a3-49c3-b597-9b6ad6c9ef5e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:54.007394 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-a7436ace-5fa4-439c-a3ef-58eb041647e2 req-b159f001-71a3-49c3-b597-9b6ad6c9ef5e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:54.020485 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-a7436ace-5fa4-439c-a3ef-58eb041647e2 req-b159f001-71a3-49c3-b597-9b6ad6c9ef5e service nova] Creating event network-changed:8d461603-b871-45af-9a83-a23e1c61b1b6 for instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d on hv-cinder-zed-3 Oct 28 11:14:54.025674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-a7436ace-5fa4-439c-a3ef-58eb041647e2 req-b159f001-71a3-49c3-b597-9b6ad6c9ef5e service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039222 Oct 28 11:14:54.027147 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 521/1055] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:14:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:54.986663 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:54.987265 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:54.993464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:54.993930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:54.994380 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:54.995056 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:54.995499 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:54.995915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:55.125843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:55.133941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0932c74d-d6bb-40b4-b87c-294b11bda53a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1364 microversion: 2.1 time: 0.151148 Oct 28 11:14:55.134533 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 535/1056] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:54 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1364 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:56.164502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:56.165284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:56.171409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:56.171884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:56.172316 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:56.173000 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:56.173451 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:56.173872 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:56.299749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:56.307951 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b2fc83f9-64be-48c2-9010-e55b5d6da77c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1364 microversion: 2.1 time: 0.148817 Oct 28 11:14:56.308511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 522/1057] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:56 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1364 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:57.288769 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:57.291080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:57.296864 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:57.297353 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:57.297792 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:57.298529 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:57.298987 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:57.299414 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:57.329832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f504020e-dc12-4251-b0fe-b9678db3a855 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045644 Oct 28 11:14:57.330608 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 536/1058] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:14:57 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:57.331972 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:57.332547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:57.339197 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:57.339765 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:57.340273 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:57.341054 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:57.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:57.342842 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:57.343457 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:57.344441 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:57.353106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:57.353616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:57.354080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:57.354790 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:57.355247 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:57.355676 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:57.366472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:14:57.389142 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-444d85cc-a6ab-4d4d-a1f6-6d7d9277f8b4 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/3cb27c8c-dc5f-4f55-a688-ede7c1ae1227" status: 200 len: 194 microversion: 2.1 time: 0.051861 Oct 28 11:14:57.389758 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 537/1059] 10.222.0.46 () {58 vars in 1413 bytes} [Fri Oct 28 11:14:57 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/3cb27c8c-dc5f-4f55-a688-ede7c1ae1227 => generated 194 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:57.482133 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:57.489910 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fee71832-35df-4ce6-8673-f1dc69cd1eab tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1364 microversion: 2.1 time: 0.162977 Oct 28 11:14:57.490504 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 523/1060] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:57 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1364 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:14:58.512783 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:58.513384 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:58.519607 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:58.520080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:58.520527 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:58.521258 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:58.521704 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:58.522156 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:58.618651 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-d5a3c685-df38-490b-bb94-2113b20630ff req-c02d0176-c1f7-49ac-b07d-900d9281c6cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d", "name": "network-vif-plugged", "status": "completed", "tag": "8d461603-b871-45af-9a83-a23e1c61b1b6"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:58.632136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-d5a3c685-df38-490b-bb94-2113b20630ff req-c02d0176-c1f7-49ac-b07d-900d9281c6cd service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:58.632927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-d5a3c685-df38-490b-bb94-2113b20630ff req-c02d0176-c1f7-49ac-b07d-900d9281c6cd service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:58.633419 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-d5a3c685-df38-490b-bb94-2113b20630ff req-c02d0176-c1f7-49ac-b07d-900d9281c6cd service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:58.644529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-d5a3c685-df38-490b-bb94-2113b20630ff req-c02d0176-c1f7-49ac-b07d-900d9281c6cd service nova] Creating event network-vif-plugged:8d461603-b871-45af-9a83-a23e1c61b1b6 for instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d on hv-cinder-zed-3 Oct 28 11:14:58.649663 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-d5a3c685-df38-490b-bb94-2113b20630ff req-c02d0176-c1f7-49ac-b07d-900d9281c6cd service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037029 Oct 28 11:14:58.650381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 524/1061] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:14:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:14:58.653858 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:58.664517 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a19ae5c3-bc41-436d-bfd7-56d31ab3dcb3 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1364 microversion: 2.1 time: 0.155444 Oct 28 11:14:58.665392 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 538/1062] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:58 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1364 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:14:58.721171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6be24c55-c624-424c-87ad-15c684342462"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:14:58.723675 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:58.731261 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:58.731819 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:58.732318 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:58.733124 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:58.733578 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:58.734029 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:58.971702 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d157fd5a-3a72-4f9a-9a74-024ac3a4ef23 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "POST /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.255479 Oct 28 11:14:58.972313 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 525/1063] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:14:58 2022] POST /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments => generated 194 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:14:59.686769 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:14:59.687406 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:14:59.693483 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:59.693967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:59.694504 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:59.695190 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:14:59.695647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:14:59.696074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:14:59.820666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:14:59.828422 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-ec4a273b-ab94-452d-81c7-b94aef631b6e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1364 microversion: 2.1 time: 0.145605 Oct 28 11:14:59.828992 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 539/1064] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:14:59 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1364 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:15:00.852688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:00.853383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:00.860025 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:00.860500 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:00.860929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:00.861608 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:00.862078 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:00.862494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:00.993815 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:15:01.001449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fb91d149-3aca-4ae1-8494-4635edcb431f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1553 microversion: 2.1 time: 0.153493 Oct 28 11:15:01.002026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 526/1065] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:15:00 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1553 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:15:01.023838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:01.024329 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:01.030201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:01.030677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:01.031108 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:01.031787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:01.032272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:01.032693 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:01.111836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-763f7ee8-e1c1-49c3-b98e-c4991bc26ac8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.091945 Oct 28 11:15:01.112436 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 540/1066] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:15:01 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-interface => generated 284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:02.008251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-729836e7-4bfc-4f2b-a08d-17c3c0999af7 req-56b66b18-3965-44d2-be3f-4f6d2d4a3420 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d", "tag": "8d461603-b871-45af-9a83-a23e1c61b1b6"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:15:02.013553 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:15:02.014240 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:15:02.021187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-729836e7-4bfc-4f2b-a08d-17c3c0999af7 req-56b66b18-3965-44d2-be3f-4f6d2d4a3420 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:02.021692 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-729836e7-4bfc-4f2b-a08d-17c3c0999af7 req-56b66b18-3965-44d2-be3f-4f6d2d4a3420 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:02.022145 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-729836e7-4bfc-4f2b-a08d-17c3c0999af7 req-56b66b18-3965-44d2-be3f-4f6d2d4a3420 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:02.033179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-729836e7-4bfc-4f2b-a08d-17c3c0999af7 req-56b66b18-3965-44d2-be3f-4f6d2d4a3420 service nova] Creating event network-changed:8d461603-b871-45af-9a83-a23e1c61b1b6 for instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d on hv-cinder-zed-3 Oct 28 11:15:02.039056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-729836e7-4bfc-4f2b-a08d-17c3c0999af7 req-56b66b18-3965-44d2-be3f-4f6d2d4a3420 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034343 Oct 28 11:15:02.039665 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 527/1067] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:15:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:06.496944 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=6 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:15:06.501017 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:06.503203 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:06.509870 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:06.510443 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:06.510881 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:06.511562 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:06.512016 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:06.512440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:06.542397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3848d821-60fd-47dd-86b2-6e51a6342169 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.046159 Oct 28 11:15:06.542882 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 541/1068] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:15:06 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments => generated 371 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:06.682074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:06.682659 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:06.689282 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:06.689757 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:06.690222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:06.690909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:06.691362 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:06.691783 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:06.720167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b0f86e8e-48da-4432-8f2e-546512cf1b50 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/3cb27c8c-dc5f-4f55-a688-ede7c1ae1227" status: 200 len: 194 microversion: 2.1 time: 0.043186 Oct 28 11:15:06.720749 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 528/1069] 10.222.0.46 () {58 vars in 1413 bytes} [Fri Oct 28 11:15:06 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/3cb27c8c-dc5f-4f55-a688-ede7c1ae1227 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:06.731147 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:06.731639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:06.737540 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:06.738045 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:06.738479 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:06.739159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:06.739611 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:06.740032 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:06.823116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lazy-loading 'services' on Instance uuid db8a46ce-9e4d-4366-949b-b4584021c35b {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:15:06.826430 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:15:06.894733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-34fc420a-15a2-4021-bf2e-fcbaa23d3a43 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/3cb27c8c-dc5f-4f55-a688-ede7c1ae1227" status: 202 len: 0 microversion: 2.1 time: 0.167368 Oct 28 11:15:06.895592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 542/1070] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:15:06 2022] DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/3cb27c8c-dc5f-4f55-a688-ede7c1ae1227 => generated 0 bytes in 169 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 28 11:15:08.014784 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:08.015276 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:08.021615 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:08.022140 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:08.022581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:08.023266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:08.023721 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:08.024176 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:08.053106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-62e33bba-7bb7-4a59-82fc-fcb7fa4d32dc tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/6be24c55-c624-424c-87ad-15c684342462" status: 200 len: 194 microversion: 2.1 time: 0.042219 Oct 28 11:15:08.053693 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 529/1071] 10.222.0.46 () {58 vars in 1413 bytes} [Fri Oct 28 11:15:08 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/6be24c55-c624-424c-87ad-15c684342462 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:08.065305 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:08.065911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:08.072766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:08.073243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:08.073674 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:08.074385 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:08.074836 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:08.075252 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:08.154606 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lazy-loading 'services' on Instance uuid db8a46ce-9e4d-4366-949b-b4584021c35b {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:15:08.223325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-53592cbb-cfa4-44f5-a44f-f8aee2234322 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/6be24c55-c624-424c-87ad-15c684342462" status: 202 len: 0 microversion: 2.1 time: 0.162194 Oct 28 11:15:08.223900 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 543/1072] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:15:08 2022] DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments/6be24c55-c624-424c-87ad-15c684342462 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:15:09.396026 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:09.398818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:09.405118 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.405601 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.406101 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.406791 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.407242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.407668 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.436104 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c361823e-2942-40d1-ad26-39483a950939 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043996 Oct 28 11:15:09.436644 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 530/1073] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:15:09 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:15:09.550899 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:09.553209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:09.555558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=57,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:15:09.561072 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.561544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.561996 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.562677 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.563124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.563679 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.594008 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c8999cbc-080f-4e4a-a6ce-1ec5e728a600 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047539 Oct 28 11:15:09.594477 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 544/1074] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:15:09 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:15:09.655410 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:09.655908 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:09.657814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=71,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:15:09.662890 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.663986 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.664423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.665110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.665563 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.666005 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.691869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:15:09.766700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e38f63e6-29ff-4849-a3f2-b20d3afb755b tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 204 len: 0 microversion: 2.1 time: 0.114724 Oct 28 11:15:09.767362 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 531/1075] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:15:09 2022] DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:15:09.776359 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:09.776961 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:09.783080 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.783554 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.783977 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.784654 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:09.785099 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:09.785516 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:09.914358 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:15:09.921915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-da78e5ca-fa49-44f6-b785-c50501560eb2 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1672 microversion: 2.1 time: 0.150005 Oct 28 11:15:09.922513 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 545/1076] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:15:09 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1672 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:15:10.945886 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:10.946626 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:10.952915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:10.953399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:10.953836 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:10.954566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:10.955027 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:10.955446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:11.087224 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:15:11.094955 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4fb435ec-ffea-4ac3-9927-cd9acb011f33 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1672 microversion: 2.1 time: 0.153329 Oct 28 11:15:11.095513 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 532/1077] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:15:10 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1672 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:15:12.120150 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:12.120832 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:12.126927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:12.127401 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:12.128050 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:12.128740 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:12.129192 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:12.129606 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:12.193393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a1f71865b51b4e8480932fffae985567', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64177183598843a68fc7fbe8257d4881', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:15:12.203819 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1ba3f766-e490-4ae3-bf7d-91906a6ce9c0 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 200 len: 1604 microversion: 2.1 time: 0.088798 Oct 28 11:15:12.204408 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 546/1078] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:15:12 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 1604 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:15:12.463120 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-9b4d6576-a63b-477e-9b1e-6035cae13523 req-833826d4-f109-41e9-a8ab-351577beb108 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "db8a46ce-9e4d-4366-949b-b4584021c35b", "name": "network-vif-deleted", "tag": "d701ecd4-e08b-4414-80c4-ca9da96dd6bc"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:15:12.475980 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-9b4d6576-a63b-477e-9b1e-6035cae13523 req-833826d4-f109-41e9-a8ab-351577beb108 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:12.476743 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-9b4d6576-a63b-477e-9b1e-6035cae13523 req-833826d4-f109-41e9-a8ab-351577beb108 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:12.477677 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-9b4d6576-a63b-477e-9b1e-6035cae13523 req-833826d4-f109-41e9-a8ab-351577beb108 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:12.495414 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-9b4d6576-a63b-477e-9b1e-6035cae13523 req-833826d4-f109-41e9-a8ab-351577beb108 service nova] Creating event network-vif-deleted:d701ecd4-e08b-4414-80c4-ca9da96dd6bc for instance db8a46ce-9e4d-4366-949b-b4584021c35b on hv-cinder-zed-3 Oct 28 11:15:12.500387 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-9b4d6576-a63b-477e-9b1e-6035cae13523 req-833826d4-f109-41e9-a8ab-351577beb108 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042614 Oct 28 11:15:12.501267 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 533/1079] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:15:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:13.227594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:13.228213 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:13.234572 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:13.235068 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:13.235502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:13.236206 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:13.236838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:13.237263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:13.255071 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] HTTP exception thrown: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Request method failure captured: Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b HTTP/1.1 Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7gfPevt3Jy22gRQNYvOjgge97akpdlKosCG4oPLI8otHxDouAaic-E4LXbphPWG70JOFkrtElTMcxuCNeP2S-mTWhhY1brMe1RcH188jj9YBxcN_PwE3wblubAQuB8v_vUTtjBSkVxncszTFpB8qQqk4G9xnq65oTjtP_Vpe4hObnM Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:15:13.256733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/64177183598843a68fc7fbe8257d4881"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_64177183598843a68fc7fbe8257d4881"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: a1f71865b51b4e8480932fffae985567 Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:13.259165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'db8a46ce-9e4d-4366-949b-b4584021c35b'} Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:15:13.260152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:13.261712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:13.263260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Returning 404 to user: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:15:13.263260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-80568cd9-45ce-4279-b99c-3b2e6c495f3e tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 404 len: 111 microversion: 2.1 time: 0.035469 Oct 28 11:15:13.263260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 547/1080] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:15:13 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:15:13.270755 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b5b3856d-0e93-451c-aa0a-813ee38ac502 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '.version_select at 0x7f42fa633ca0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:13.290106 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b5b3856d-0e93-451c-aa0a-813ee38ac502 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1622864765" status: 202 len: 0 microversion: 2.1 time: 0.025547 Oct 28 11:15:13.290684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 534/1081] 10.222.0.46 () {58 vars in 1221 bytes} [Fri Oct 28 11:15:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1622864765 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:15:16.152180 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:16.152673 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:16.158508 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:16.158989 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:16.159420 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:16.160112 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:16.160558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:16.160979 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:16.174186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] HTTP exception thrown: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Request method failure captured: Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b HTTP/1.1 Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7gfPevt3Jy22gRQNYvOjgge97akpdlKosCG4oPLI8otHxDouAaic-E4LXbphPWG70JOFkrtElTMcxuCNeP2S-mTWhhY1brMe1RcH188jj9YBxcN_PwE3wblubAQuB8v_vUTtjBSkVxncszTFpB8qQqk4G9xnq65oTjtP_Vpe4hObnM Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:15:16.175687 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/64177183598843a68fc7fbe8257d4881"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_64177183598843a68fc7fbe8257d4881"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: a1f71865b51b4e8480932fffae985567 Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.177253 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'db8a46ce-9e4d-4366-949b-b4584021c35b'} Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:15:16.178097 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.179331 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.181123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Returning 404 to user: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:15:16.181123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-926951f1-7c0c-40cf-843c-b9d07d09c095 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 404 len: 111 microversion: 2.1 time: 0.028920 Oct 28 11:15:16.181123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 548/1082] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:15:16 2022] DELETE /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:15:16.186752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:16.187520 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: db8a46ce-9e4d-4366-949b-b4584021c35b] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:16.194006 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:16.194501 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:16.194935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:16.196266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:16.196721 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:16.197146 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:16.209790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] HTTP exception thrown: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Request method failure captured: Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b HTTP/1.1 Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7gfPevt3Jy22gRQNYvOjgge97akpdlKosCG4oPLI8otHxDouAaic-E4LXbphPWG70JOFkrtElTMcxuCNeP2S-mTWhhY1brMe1RcH188jj9YBxcN_PwE3wblubAQuB8v_vUTtjBSkVxncszTFpB8qQqk4G9xnq65oTjtP_Vpe4hObnM Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:15:16.211494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/64177183598843a68fc7fbe8257d4881"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_64177183598843a68fc7fbe8257d4881"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: a1f71865b51b4e8480932fffae985567 Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.213083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'db8a46ce-9e4d-4366-949b-b4584021c35b'} Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:15:16.214035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. Oct 28 11:15:16.215523 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:16.217051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Returning 404 to user: Instance db8a46ce-9e4d-4366-949b-b4584021c35b could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:15:16.217051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-cc0204c3-fe07-4c78-97ae-b7c4ff2c5e27 tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b" status: 404 len: 111 microversion: 2.1 time: 0.030533 Oct 28 11:15:16.217051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 535/1083] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:15:16 2022] GET /compute/v2.1/servers/db8a46ce-9e4d-4366-949b-b4584021c35b => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:15:17.350773 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:17.351255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:17.357703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:17.358224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:17.358662 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:17.359351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:17.359802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:17.360224 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:17.374179 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] HTTP exception thrown: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Request method failure captured: Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 HTTP/1.1 Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7gfPevt3Jy22gRQNYvOjgge97akpdlKosCG4oPLI8otHxDouAaic-E4LXbphPWG70JOFkrtElTMcxuCNeP2S-mTWhhY1brMe1RcH188jj9YBxcN_PwE3wblubAQuB8v_vUTtjBSkVxncszTFpB8qQqk4G9xnq65oTjtP_Vpe4hObnM Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:15:17.375694 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/64177183598843a68fc7fbe8257d4881"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_64177183598843a68fc7fbe8257d4881"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: a1f71865b51b4e8480932fffae985567 Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.377296 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'e4432177-31d4-43c8-a6a2-a1d727b144a2'} Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:15:17.378400 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.379915 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.381543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Returning 404 to user: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:15:17.381543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8b9d7bb2-36e2-481a-bebf-02cf50a869ec tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "DELETE /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 404 len: 111 microversion: 2.1 time: 0.030191 Oct 28 11:15:17.381543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 549/1084] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:15:17 2022] DELETE /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:15:17.385292 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:17.385906 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] [instance: e4432177-31d4-43c8-a6a2-a1d727b144a2] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:15:17.392509 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:17.392997 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:17.393433 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:17.394174 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:15:17.394636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:15:17.395068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:15:17.408195 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] HTTP exception thrown: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Request method failure captured: Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 HTTP/1.1 Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7gfPevt3Jy22gRQNYvOjgge97akpdlKosCG4oPLI8otHxDouAaic-E4LXbphPWG70JOFkrtElTMcxuCNeP2S-mTWhhY1brMe1RcH188jj9YBxcN_PwE3wblubAQuB8v_vUTtjBSkVxncszTFpB8qQqk4G9xnq65oTjtP_Vpe4hObnM Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:15:17.409718 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/64177183598843a68fc7fbe8257d4881"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_64177183598843a68fc7fbe8257d4881"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/64177183598843a68fc7fbe8257d4881"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 64177183598843a68fc7fbe8257d4881 Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeTestJSON-217162543 Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: a1f71865b51b4e8480932fffae985567 Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeTestJSON-217162543-project Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.411343 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'e4432177-31d4-43c8-a6a2-a1d727b144a2'} Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:15:17.412384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. Oct 28 11:15:17.413887 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:15:17.415948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] Returning 404 to user: Instance e4432177-31d4-43c8-a6a2-a1d727b144a2 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:15:17.415948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3aa01fcb-1032-4f3a-8ec6-f91600b4463a tempest-AttachVolumeTestJSON-217162543 tempest-AttachVolumeTestJSON-217162543-project] 10.222.0.46 "GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2" status: 404 len: 111 microversion: 2.1 time: 0.029675 Oct 28 11:15:17.415948 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 536/1085] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:15:17 2022] GET /compute/v2.1/servers/e4432177-31d4-43c8-a6a2-a1d727b144a2 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:15:21.691860 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-596d36e1-3d33-4dec-8e32-b45da9c1db79 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Action: 'create', calling method: .version_select at 0x7f42fa697dc0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1840750078", "metadata": {"Type": "work"}}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:15:21.974027 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-596d36e1-3d33-4dec-8e32-b45da9c1db79 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] 10.222.0.46 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.392201 Oct 28 11:15:21.974651 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 550/1086] 10.222.0.46 () {60 vars in 1156 bytes} [Fri Oct 28 11:15:21 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 394 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 28 11:15:21.987861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-eec4a308-9cd4-4c78-b685-f7f95d183862 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Calling method '.version_select at 0x7f42fa862b80>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:22.038446 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-eec4a308-9cd4-4c78-b685-f7f95d183862 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736" status: 200 len: 353 microversion: 2.1 time: 0.054680 Oct 28 11:15:22.039012 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 537/1087] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:15:21 2022] GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736 => generated 353 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:23.052767 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3e038bca-03f5-4b59-a1ef-d1078d4256ca tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Calling method '.version_select at 0x7f42fa5f04c0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:23.097822 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3e038bca-03f5-4b59-a1ef-d1078d4256ca tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736" status: 200 len: 354 microversion: 2.1 time: 0.049046 Oct 28 11:15:23.098434 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 551/1088] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:15:23 2022] GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736 => generated 354 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:23.112976 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-65226dc6-9472-4f65-880a-bcd291f49fd6 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Calling method '.version_select at 0x7f42faa8ca60>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:23.158618 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-65226dc6-9472-4f65-880a-bcd291f49fd6 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736" status: 200 len: 354 microversion: 2.1 time: 0.050002 Oct 28 11:15:23.159179 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 538/1089] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:15:23 2022] GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736 => generated 354 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:15:23.178008 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-cf0a82f6-efd1-46a6-a295-f4c39db53736 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Calling method '.version_select at 0x7f42fa93cdc0>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:23.237347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-cf0a82f6-efd1-46a6-a295-f4c39db53736 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] 10.222.0.46 "DELETE /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736" status: 202 len: 0 microversion: 2.1 time: 0.063063 Oct 28 11:15:23.238029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 552/1090] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:15:23 2022] DELETE /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736 => generated 0 bytes in 64 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:15:23.246541 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ca986e19-15fe-4f66-bc36-3c5b2e60cb2c tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Calling method '.version_select at 0x7f42fa78c430>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:23.295040 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ca986e19-15fe-4f66-bc36-3c5b2e60cb2c tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736" status: 200 len: 353 microversion: 2.1 time: 0.052881 Oct 28 11:15:23.295694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 539/1091] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:15:23 2022] GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736 => generated 353 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:15:24.310079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-500fe97e-8872-4b6f-bb61-7e96dde8baa1 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Calling method '.version_select at 0x7f42fa615940>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:15:24.337271 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-500fe97e-8872-4b6f-bb61-7e96dde8baa1 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] HTTP exception thrown: Volume 65b9894b-f573-4fb4-b2d3-a72df4c0d736 could not be found. Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-500fe97e-8872-4b6f-bb61-7e96dde8baa1 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Request method failure captured: Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736 HTTP/1.1 Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7nJpCFc85dIHjrtqXWU7d0HnZ9R7bPfCmM6ZAEsmeKaFgnA-FP2lV7nfQqlEGBgPTs-OCEFDiPei-DNtKifrTKevkt3KD9u6FqlrpZRbL36FeIjNxlFWW-UTC9VwUX99_J9L7I-i6xAdatwRTXMSp7FL20mHBXHbpy7wQE2oSlctE8 Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: b49ad4a760764797a6c6cf253facde70 Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesGetTestJSON-1961938520 Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:15:24.339569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/b49ad4a760764797a6c6cf253facde70"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_b49ad4a760764797a6c6cf253facde70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/b49ad4a760764797a6c6cf253facde70"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/b49ad4a760764797a6c6cf253facde70"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesGetTestJSON-1961938520 Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: b49ad4a760764797a6c6cf253facde70 Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesGetTestJSON-1961938520 Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesGetTestJSON-1961938520-project Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: 6454e73eeb3b4e8a87090a92308f7ecc Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesGetTestJSON-1961938520-project Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Volume 65b9894b-f573-4fb4-b2d3-a72df4c0d736 could not be found. Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '65b9894b-f573-4fb4-b2d3-a72df4c0d736'} Oct 28 11:15:24.341058 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 65b9894b-f573-4fb4-b2d3-a72df4c0d736 could not be found. (HTTP 404) (Request-ID: req-9337b2e8-a14e-432e-be7d-8fba587b5dae) Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:24.342222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:15:24.343438 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 65b9894b-f573-4fb4-b2d3-a72df4c0d736 could not be found. Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 11:15:24.344609 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 65b9894b-f573-4fb4-b2d3-a72df4c0d736 could not be found. Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 65b9894b-f573-4fb4-b2d3-a72df4c0d736 could not be found. Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-500fe97e-8872-4b6f-bb61-7e96dde8baa1 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] Returning 404 to user: Volume 65b9894b-f573-4fb4-b2d3-a72df4c0d736 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-500fe97e-8872-4b6f-bb61-7e96dde8baa1 tempest-VolumesGetTestJSON-1961938520 tempest-VolumesGetTestJSON-1961938520-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736" status: 404 len: 109 microversion: 2.1 time: 0.035522 Oct 28 11:15:24.345930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 553/1092] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:15:24 2022] GET /compute/v2.1/os-volumes/65b9894b-f573-4fb4-b2d3-a72df4c0d736 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:16:06.284302 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=17,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:06.285086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=67,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:06.289326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1789942325", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "networks": [{"uuid": "4dbdb288-c7d5-4203-bd75-39f48212d6ac"}]}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:06.342694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.network.neutron [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] validate_networks() for [('4dbdb288-c7d5-4203-bd75-39f48212d6ac', None, None, None, None, None)] {{(pid=120544) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:16:06.684929 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Flavor limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:16:06.685472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Image limits 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:16:06.686713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Flavor pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:16:06.687230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Image pref 0:0:0 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:16:06.687759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.virt.hardware [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120544) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:16:06.699069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:06.699646 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:06.700184 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:06.701031 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:06.701681 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:06.702252 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:06.718940 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Getting quotas for project 33af503fc7e9410885f704ffbfcb900b. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:16:06.722907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Getting quotas for user c689e56cfc3f43e187c8804fd8577cb7 and project 33af503fc7e9410885f704ffbfcb900b. Resources: {'cores', 'instances', 'ram'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:16:06.734295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Going to run 1 instances... {{(pid=120544) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:16:06.736572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120544) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:16:06.782247 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-afd3ee88-a50e-4a4e-b74e-d6d39987afff tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.499277 Oct 28 11:16:06.782886 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 540/1093] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:16:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 501 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:16:06.790567 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=11 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:06.791494 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=34,nova_cell1:INSERT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:06.796043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-06faa8a1-fc0a-4ae4-85bf-368ca67f9847 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:06.796644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-06faa8a1-fc0a-4ae4-85bf-368ca67f9847 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:06.852754 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-06faa8a1-fc0a-4ae4-85bf-368ca67f9847 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:06.860137 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-06faa8a1-fc0a-4ae4-85bf-368ca67f9847 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1219 microversion: 2.1 time: 0.070344 Oct 28 11:16:06.860701 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 554/1094] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:06 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1219 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:07.883319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c07b86c9-48c9-4046-bd3b-6c6711a98e58 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:07.883936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c07b86c9-48c9-4046-bd3b-6c6711a98e58 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:07.934778 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c07b86c9-48c9-4046-bd3b-6c6711a98e58 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:07.942949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c07b86c9-48c9-4046-bd3b-6c6711a98e58 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1219 microversion: 2.1 time: 0.064020 Oct 28 11:16:07.943503 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 541/1095] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:07 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1219 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:08.965518 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-672c08ce-ccdf-4cb8-8a90-4a1e9b34ed10 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:08.966152 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-672c08ce-ccdf-4cb8-8a90-4a1e9b34ed10 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:09.019511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-672c08ce-ccdf-4cb8-8a90-4a1e9b34ed10 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:09.026659 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-672c08ce-ccdf-4cb8-8a90-4a1e9b34ed10 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1219 microversion: 2.1 time: 0.064812 Oct 28 11:16:09.027225 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 555/1096] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:08 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1219 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:10.049430 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e3bdf709-60f4-4650-8ee8-bbc52b025470 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:10.050068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e3bdf709-60f4-4650-8ee8-bbc52b025470 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:10.103892 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e3bdf709-60f4-4650-8ee8-bbc52b025470 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:10.111335 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e3bdf709-60f4-4650-8ee8-bbc52b025470 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1219 microversion: 2.1 time: 0.065756 Oct 28 11:16:10.111911 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 542/1097] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:10 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1219 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:11.135132 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:11.135735 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:11.141702 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:11.142209 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:11.142639 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:11.143318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:11.143764 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:11.144178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:11.203311 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:11.210851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-14966842-3cda-43dc-881e-4ca7d424b5b2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1223 microversion: 2.1 time: 0.079581 Oct 28 11:16:11.211403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 556/1098] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:11 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1223 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:12.233998 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:12.235227 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:12.242069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:12.242554 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:12.242989 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:12.243947 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:12.244396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:12.244821 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:12.322191 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:12.330295 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a657ff56-0e6f-4b23-9b81-0caf857f3ec6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1277 microversion: 2.1 time: 0.100341 Oct 28 11:16:12.330861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 543/1099] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:12 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1277 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:13.350970 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-e46883fa-327d-47cc-9edd-bd494d67aebc req-2ee969c2-ace4-4a8c-8735-fa5965ffcedc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3ad6871e-a8cc-443d-ab6c-ef650207f988", "tag": "ac892418-282c-4436-8152-650c1baa89c6"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:13.352511 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:13.353287 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:13.360166 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:13.360949 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:13.361607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:13.362656 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:13.363461 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:13.364126 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:13.368838 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e46883fa-327d-47cc-9edd-bd494d67aebc req-2ee969c2-ace4-4a8c-8735-fa5965ffcedc service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:13.369314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e46883fa-327d-47cc-9edd-bd494d67aebc req-2ee969c2-ace4-4a8c-8735-fa5965ffcedc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:13.369745 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e46883fa-327d-47cc-9edd-bd494d67aebc req-2ee969c2-ace4-4a8c-8735-fa5965ffcedc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:13.381820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-e46883fa-327d-47cc-9edd-bd494d67aebc req-2ee969c2-ace4-4a8c-8735-fa5965ffcedc service nova] Creating event network-changed:ac892418-282c-4436-8152-650c1baa89c6 for instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 on hv-cinder-zed-3 Oct 28 11:16:13.387326 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-e46883fa-327d-47cc-9edd-bd494d67aebc req-2ee969c2-ace4-4a8c-8735-fa5965ffcedc service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039942 Oct 28 11:16:13.388293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 557/1100] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:16:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:13.599746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:13.607333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-67acfdc0-09b2-4d3e-b5ef-398bddd18fc9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1319 microversion: 2.1 time: 0.258714 Oct 28 11:16:13.607908 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 544/1101] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:13 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1319 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:14.632620 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:14.633313 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:14.639538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:14.640007 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:14.640433 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:14.641102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:14.641546 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:14.641958 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:14.766644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:14.774188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0c19fa1d-079c-4e27-af05-4e1ed741a0a6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1319 microversion: 2.1 time: 0.145747 Oct 28 11:16:14.774778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 558/1102] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:14 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1319 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:15.797234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:15.797843 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:15.804502 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:15.804982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:15.805411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:15.806138 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:15.806589 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:15.807014 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:16.022498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:16.030746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-6c759020-2589-4249-89e7-d2a3808c53db tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1319 microversion: 2.1 time: 0.237958 Oct 28 11:16:16.031388 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 545/1103] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:15 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1319 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:17.053678 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:17.054330 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:17.060237 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:17.060726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:17.061804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:17.062526 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:17.062985 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:17.063411 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:17.193697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:17.201523 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-12ef00a4-5bb2-4ed1-af07-24e5c267020f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1319 microversion: 2.1 time: 0.151685 Oct 28 11:16:17.202118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 559/1104] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:17 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:17.256595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:17.257154 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:17.258907 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:17.264543 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:17.265017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:17.265443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:17.266169 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:17.266622 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:17.267042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:17.289751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-32397923-29fe-456f-852a-e8c61befbd8e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/ips" status: 200 len: 149 microversion: 2.1 time: 0.037006 Oct 28 11:16:17.290440 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 546/1105] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:16:17 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/ips => generated 149 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:18.224377 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:18.225017 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:18.230840 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:18.231319 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:18.231748 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:18.232429 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:18.232875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:18.233287 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:18.357685 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:18.365102 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-575ddbfa-c6d9-4be0-9bf6-04d79e293886 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1319 microversion: 2.1 time: 0.144683 Oct 28 11:16:18.365668 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 560/1106] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:18 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1319 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:18.619435 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a27d9de2-cd5c-49d9-94bf-6edd0777a9a2"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:18.621879 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:18.628946 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:18.629425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:18.629861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:18.630580 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:18.631034 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:18.631459 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:18.870023 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dedf9954-4f7d-48b8-aef4-6d3a4898de65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.254604 Oct 28 11:16:18.870896 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 547/1107] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:16:18 2022] POST /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments => generated 194 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:19.058798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-b4d17375-7efa-4283-a035-c42ac91ba5b6 req-744f88c8-8bfa-43cf-9815-a0dbbe559e8d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3ad6871e-a8cc-443d-ab6c-ef650207f988", "name": "network-vif-plugged", "status": "completed", "tag": "ac892418-282c-4436-8152-650c1baa89c6"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:19.063778 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:19.069847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-b4d17375-7efa-4283-a035-c42ac91ba5b6 req-744f88c8-8bfa-43cf-9815-a0dbbe559e8d service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:19.070348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-b4d17375-7efa-4283-a035-c42ac91ba5b6 req-744f88c8-8bfa-43cf-9815-a0dbbe559e8d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:19.070941 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-b4d17375-7efa-4283-a035-c42ac91ba5b6 req-744f88c8-8bfa-43cf-9815-a0dbbe559e8d service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:19.082121 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-b4d17375-7efa-4283-a035-c42ac91ba5b6 req-744f88c8-8bfa-43cf-9815-a0dbbe559e8d service nova] Creating event network-vif-plugged:ac892418-282c-4436-8152-650c1baa89c6 for instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 on hv-cinder-zed-3 Oct 28 11:16:19.087325 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-b4d17375-7efa-4283-a035-c42ac91ba5b6 req-744f88c8-8bfa-43cf-9815-a0dbbe559e8d service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032196 Oct 28 11:16:19.087938 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 561/1108] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:16:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:16:19.389460 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:19.390152 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:19.396855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:19.397332 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:19.397761 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:19.398476 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:19.398923 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:19.399341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:19.528780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:19.536754 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7f20eff7-314b-4ce2-8374-45b529ed64e6 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1319 microversion: 2.1 time: 0.152082 Oct 28 11:16:19.537341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 548/1109] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:19 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:20.559933 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:20.560532 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:20.566695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:20.567165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:20.567592 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:20.568264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:20.568709 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:20.569129 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:20.698982 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:20.706547 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-548a9dea-bd36-498b-944d-a6035bf2f05f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1319 microversion: 2.1 time: 0.150955 Oct 28 11:16:20.707104 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 562/1110] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:20 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:21.728829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:21.729441 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:21.731874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=10 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:21.737807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:21.738322 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:21.738755 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:21.739443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:21.739895 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:21.740317 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:21.866783 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:21.875494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0f05d5e2-bccb-4963-9e1f-87a05273bf92 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1508 microversion: 2.1 time: 0.150714 Oct 28 11:16:21.876068 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 549/1111] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:21 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1508 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:26.416490 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a27d9de2-cd5c-49d9-94bf-6edd0777a9a2"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:26.419118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:26.425850 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:26.426368 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:26.426804 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:26.427511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:26.427973 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:26.428397 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:26.507188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Invalid volume: volume a27d9de2-cd5c-49d9-94bf-6edd0777a9a2 is already attached to instances: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: POST /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments HTTP/1.1 Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 74 Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:16:26.508923 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {"volumeAttachment": {"volumeId": "a27d9de2-cd5c-49d9-94bf-6edd0777a9a2"}} Oct 28 11:16:26.510715 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Invalid volume: volume a27d9de2-cd5c-49d9-94bf-6edd0777a9a2 is already attached to instances: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'server_id': 'b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d', 'body': {'volumeAttachment': {'volumeId': 'a27d9de2-cd5c-49d9-94bf-6edd0777a9a2'}}} Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume a27d9de2-cd5c-49d9-94bf-6edd0777a9a2 is already attached to instances: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:16:26.511500 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume a27d9de2-cd5c-49d9-94bf-6edd0777a9a2 is already attached to instances: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:16:26.512739 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume a27d9de2-cd5c-49d9-94bf-6edd0777a9a2 is already attached to instances: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 400 to user: Invalid volume: volume a27d9de2-cd5c-49d9-94bf-6edd0777a9a2 is already attached to instances: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c72826bc-a5ed-4964-8a69-b4f390cc8d9f tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.098115 Oct 28 11:16:26.514030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 563/1112] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:16:26 2022] POST /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments => generated 174 bytes in 99 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:16:26.577779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:26.579301 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:26.586288 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:26.586801 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:26.587230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:26.587931 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:26.588379 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:26.588855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:26.669438 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lazy-loading 'services' on Instance uuid b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:16:26.780796 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c230d32c-86ab-4919-b7a0-95b90375c284 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments/a27d9de2-cd5c-49d9-94bf-6edd0777a9a2" status: 202 len: 0 microversion: 2.1 time: 0.208124 Oct 28 11:16:26.781525 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 550/1113] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:16:26 2022] DELETE /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments/a27d9de2-cd5c-49d9-94bf-6edd0777a9a2 => generated 0 bytes in 210 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:16:26.789901 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:26.792103 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:26.799201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:26.799685 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:26.800418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:26.801159 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:26.801785 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:26.802261 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:26.834373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-0dcaf372-5c5b-43ac-887c-6e7044bd9327 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048528 Oct 28 11:16:26.834927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 564/1114] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:16:26 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:16:27.848201 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:27.850541 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:27.856636 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:27.857109 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:27.857535 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:27.858311 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:27.858758 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:27.859173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:27.890868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ea332c08-84a1-4e98-9a57-7e975a6fe6db tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047543 Oct 28 11:16:27.891456 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 551/1115] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:16:27 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:16:27.955963 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:27.956543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:27.963909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:27.964605 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:27.965155 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:27.966003 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:27.966630 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:27.967140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:27.993967 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:16:28.066236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dcef445b-15e2-475b-9c3d-704e2cec26f7 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 204 len: 0 microversion: 2.1 time: 0.113945 Oct 28 11:16:28.067039 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 565/1116] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:16:27 2022] DELETE /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:16:28.241936 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:28.242604 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:28.248927 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:28.249398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:28.249864 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:28.251074 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:28.251520 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:28.251937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:28.382414 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:28.390111 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ca6f2e53-2d7c-40b0-8d88-8927e4b79d74 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1678 microversion: 2.1 time: 0.318890 Oct 28 11:16:28.390665 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 552/1117] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:28 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1678 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:29.414534 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:29.415233 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:29.421268 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:29.421742 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:29.422218 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:29.423501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:29.423949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:29.424369 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:29.556847 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:29.564491 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-1ced188f-815c-49de-a5ae-e85a4a434eb9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1678 microversion: 2.1 time: 0.154299 Oct 28 11:16:29.565052 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 566/1118] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:29 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1678 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:30.588444 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:30.589056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:30.598135 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:30.598884 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:30.600570 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:30.601320 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:30.602075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:30.602501 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:30.674374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:30.681987 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-afb6d2c6-05b8-43f8-a462-1e0c9b4e642e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 200 len: 1611 microversion: 2.1 time: 0.097736 Oct 28 11:16:30.682789 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 553/1119] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:30 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 1611 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:30.895123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-0aa31bae-fcd6-46ab-805f-45cf4bfa47b3 req-841df78f-2c4c-4ee1-9f87-ae7192217c18 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d", "name": "network-vif-deleted", "tag": "8d461603-b871-45af-9a83-a23e1c61b1b6"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:30.929613 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-0aa31bae-fcd6-46ab-805f-45cf4bfa47b3 req-841df78f-2c4c-4ee1-9f87-ae7192217c18 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:30.930848 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-0aa31bae-fcd6-46ab-805f-45cf4bfa47b3 req-841df78f-2c4c-4ee1-9f87-ae7192217c18 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:30.931381 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-0aa31bae-fcd6-46ab-805f-45cf4bfa47b3 req-841df78f-2c4c-4ee1-9f87-ae7192217c18 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:30.944307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-0aa31bae-fcd6-46ab-805f-45cf4bfa47b3 req-841df78f-2c4c-4ee1-9f87-ae7192217c18 service nova] Creating event network-vif-deleted:8d461603-b871-45af-9a83-a23e1c61b1b6 for instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d on hv-cinder-zed-3 Oct 28 11:16:30.950633 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-0aa31bae-fcd6-46ab-805f-45cf4bfa47b3 req-841df78f-2c4c-4ee1-9f87-ae7192217c18 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062582 Oct 28 11:16:30.951952 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 567/1120] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:16:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:31.705383 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:31.706151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:31.712507 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:31.712981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:31.713421 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:31.714139 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:31.714589 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:31.715007 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:31.729941 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d HTTP/1.1 Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:16:31.731712 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:16:31.733330 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d'} Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:16:31.734412 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:16:31.735917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:16:31.737086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:16:31.737086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-643cdb95-61b5-4566-982e-da02adfb4e41 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 404 len: 111 microversion: 2.1 time: 0.032285 Oct 28 11:16:31.737086 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 554/1121] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:31 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:16:31.745582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-fe43e5f6-98b0-42a4-b384-2beac9f3c010 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '.version_select at 0x7f42fbe79430>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:31.771126 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-fe43e5f6-98b0-42a4-b384-2beac9f3c010 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-43803389" status: 202 len: 0 microversion: 2.1 time: 0.032547 Oct 28 11:16:31.772266 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 568/1122] 10.222.0.46 () {58 vars in 1215 bytes} [Fri Oct 28 11:16:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-43803389 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:16:32.382231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-da22d22d-f302-4803-bda0-c062955d7409 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: .version_select at 0x7f42fa59edc0>, body: {"keypair": {"name": "tempest-keypair-503492479", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBfNFDGNl6jhDZHRRxpJhfw9jtGsP2ejmBRyLrugOdEAbwq7BM8aV1RVS8n6gYf0vRiGEn1i4shNPP3cRYgivJ7f6111P8LmYctsaNrrOhqo2y6pfm8amBe63roeK0tBUA=="}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:32.397397 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-da22d22d-f302-4803-bda0-c062955d7409 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:16:32.401281 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.quota [None req-da22d22d-f302-4803-bda0-c062955d7409 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'key_pairs'} {{(pid=120544) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:16:32.422753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-da22d22d-f302-4803-bda0-c062955d7409 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.044490 Oct 28 11:16:32.423874 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 555/1123] 10.222.0.46 () {60 vars in 1159 bytes} [Fri Oct 28 11:16:32 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:33.604726 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-33970008", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "447754b6-a798-4b6b-beaa-8e9ec6ea9fd5"}], "security_groups": [{"name": "tempest-securitygroup--932915824"}], "key_name": "tempest-keypair-503492479", "user_data": "}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:33.807312 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] validate_networks() for [('447754b6-a798-4b6b-beaa-8e9ec6ea9fd5', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:16:34.170254 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:16:34.170674 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:16:34.171141 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:16:34.171549 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:16:34.171949 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:16:34.180728 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:34.181204 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:34.181631 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:34.182390 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:34.182841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:34.183263 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:34.200202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:16:34.204167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Getting quotas for user ff4b7731f96643d0a24e63d31f5f6f00 and project 68345feb30fe454083122ecaf73ef747. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:16:34.216582 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:16:34.221087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:16:34.268293 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR oslo.messaging._drivers.impl_rabbit [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [e9f3faf0-c492-4eb3-9c77-2dd5ccacca52] AMQP server on 10.222.0.46:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 28 11:16:35.275241 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO oslo.messaging._drivers.impl_rabbit [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [e9f3faf0-c492-4eb3-9c77-2dd5ccacca52] Reconnected to AMQP server on 10.222.0.46:5672 via [amqp] client with port 36472. Oct 28 11:16:35.280393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-7c4bca93-282e-4312-9c5a-e82190493b2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.680016 Oct 28 11:16:35.281041 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 569/1124] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:16:33 2022] POST /compute/v2.1/servers => generated 401 bytes in 1681 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:16:35.293265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-98c9bfa8-b55c-46d4-8496-5170aaf07c65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:35.293965 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-98c9bfa8-b55c-46d4-8496-5170aaf07c65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:35.412130 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-98c9bfa8-b55c-46d4-8496-5170aaf07c65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:35.419975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-98c9bfa8-b55c-46d4-8496-5170aaf07c65 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1239 microversion: 2.1 time: 0.130997 Oct 28 11:16:35.420547 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 556/1125] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:35 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1239 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:36.443043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:36.443759 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:36.450142 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:36.450632 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:36.451075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:36.451776 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:36.452239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:36.452667 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:36.516123 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:36.524394 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e851bde6-7900-424d-8910-116c081d30f1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1297 microversion: 2.1 time: 0.085294 Oct 28 11:16:36.524905 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 570/1126] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:36 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1297 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:37.548752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:37.549368 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:37.551840 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=4 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:37.559297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:37.560164 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:37.560974 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:37.562275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:37.563100 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:37.564405 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:37.569811 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:37.710234 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-1f579fc5-fd78-4ffb-89e7-b6991e194c67 req-14882d76-891b-418f-b0c3-bffd1d0bf452 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cd5eee5e-9d7f-40b0-abd1-0393d82f90ad", "tag": "a924d329-0af3-4eae-b277-6e7253359d9f"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:37.716712 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:37.723937 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-1f579fc5-fd78-4ffb-89e7-b6991e194c67 req-14882d76-891b-418f-b0c3-bffd1d0bf452 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:37.724732 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-1f579fc5-fd78-4ffb-89e7-b6991e194c67 req-14882d76-891b-418f-b0c3-bffd1d0bf452 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:37.725216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-1f579fc5-fd78-4ffb-89e7-b6991e194c67 req-14882d76-891b-418f-b0c3-bffd1d0bf452 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:37.730541 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:37.739594 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-1f579fc5-fd78-4ffb-89e7-b6991e194c67 req-14882d76-891b-418f-b0c3-bffd1d0bf452 service nova] Creating event network-changed:a924d329-0af3-4eae-b277-6e7253359d9f for instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad on hv-cinder-zed-3 Oct 28 11:16:37.745114 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-1f579fc5-fd78-4ffb-89e7-b6991e194c67 req-14882d76-891b-418f-b0c3-bffd1d0bf452 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040579 Oct 28 11:16:37.745925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 571/1127] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:16:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:37.794763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:37.803170 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3036757c-3acf-4a74-958f-df378ebbe98b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.259000 Oct 28 11:16:37.803745 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 557/1128] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:37 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:38.826988 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:38.827690 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:38.836264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:38.836749 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:38.837186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:38.837906 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:38.838393 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:38.838820 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:38.966765 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:38.975079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b31d0f6a-42be-49f2-96aa-c06e1c4cb77e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.152794 Oct 28 11:16:38.975647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 572/1129] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:38 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:39.997351 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:39.998011 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:40.004768 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:40.005244 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:40.005676 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:40.006387 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:40.006843 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:40.007264 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:40.132447 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:40.140350 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-acb20ea3-245a-425c-b969-713e8b215738 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.147115 Oct 28 11:16:40.140970 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 558/1130] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:39 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:41.162686 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:41.163299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:41.169246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:41.169725 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:41.170188 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:41.170870 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:41.171320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:41.171741 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:41.298664 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:41.306222 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2192f6e1-6632-4e88-9c89-31104c7ac3f8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.147421 Oct 28 11:16:41.306789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 573/1131] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:41 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 28 11:16:42.329089 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:42.330356 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:42.336777 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:42.337256 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:42.337693 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:42.338409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:42.338861 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:42.339281 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:42.472993 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:42.480763 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c81a140f-70ef-464e-8e20-430e9f141809 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.155494 Oct 28 11:16:42.481333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 559/1132] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:42 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:43.417029 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "51f24d1f-2969-42a4-a002-cfded573d8d9", "device": "/dev/sdb"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:43.419943 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:43.426360 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:43.426839 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:43.427270 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:43.427980 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:43.428428 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:43.428850 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:43.506333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:43.507092 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:43.513838 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:43.514396 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:43.514837 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:43.515554 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:43.516010 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:43.516468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:43.663465 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:43.671230 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3fea92b3-e4ad-4c29-878d-731ea6446527 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.170134 Oct 28 11:16:43.671792 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 560/1133] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:43 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:43.704680 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-03563ef1-a649-4ec5-bb76-f38594064f0c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "POST /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.291503 Oct 28 11:16:43.705318 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 574/1134] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:16:43 2022] POST /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988/os-volume_attachments => generated 194 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:44.695531 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:44.696078 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:44.702537 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:44.703014 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:44.703447 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:44.704134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:44.704582 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:44.705003 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:44.835162 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:44.842756 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-3f80f501-4068-42ae-9b64-7a36fc47f624 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.151179 Oct 28 11:16:44.843314 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 561/1135] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:44 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:45.103908 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-e5fb762a-173a-43d7-b536-001a0098ce0b req-12ecb501-345b-43af-9d2f-d32ce464a2bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cd5eee5e-9d7f-40b0-abd1-0393d82f90ad", "name": "network-vif-plugged", "status": "completed", "tag": "a924d329-0af3-4eae-b277-6e7253359d9f"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:45.109455 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:45.110240 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=6 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:45.116207 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e5fb762a-173a-43d7-b536-001a0098ce0b req-12ecb501-345b-43af-9d2f-d32ce464a2bc service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:45.116688 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e5fb762a-173a-43d7-b536-001a0098ce0b req-12ecb501-345b-43af-9d2f-d32ce464a2bc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:45.117157 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-e5fb762a-173a-43d7-b536-001a0098ce0b req-12ecb501-345b-43af-9d2f-d32ce464a2bc service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:45.129624 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-e5fb762a-173a-43d7-b536-001a0098ce0b req-12ecb501-345b-43af-9d2f-d32ce464a2bc service nova] Creating event network-vif-plugged:a924d329-0af3-4eae-b277-6e7253359d9f for instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad on hv-cinder-zed-3 Oct 28 11:16:45.134373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-e5fb762a-173a-43d7-b536-001a0098ce0b req-12ecb501-345b-43af-9d2f-d32ce464a2bc service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034609 Oct 28 11:16:45.134959 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 575/1136] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:16:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:45.866309 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:45.866926 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:45.870013 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=6 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:16:45.874265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:45.874752 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:45.875185 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:45.875873 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:45.876326 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:45.876750 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:46.003156 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:46.010722 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-ad8911cc-af4e-4473-81da-5232e50518d8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1364 microversion: 2.1 time: 0.148887 Oct 28 11:16:46.011284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 562/1137] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:45 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1364 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:47.033650 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:47.034370 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:47.040930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:47.041437 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:47.041869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:47.042580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:47.043031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:47.043447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:47.172542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:16:47.180672 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b7e3f03b-7fd3-44ec-9ad2-b062441630b8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1554 microversion: 2.1 time: 0.151678 Oct 28 11:16:47.181243 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 576/1138] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:16:47 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1554 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:16:47.203496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:16:47.204021 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:16:47.210492 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:47.210975 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:47.211411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:47.212100 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:47.212555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:47.212982 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:47.288392 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4f4e1305-d13d-4c43-9c65-e4051dd38d0b tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.088635 Oct 28 11:16:47.289017 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 563/1139] 10.222.0.46 () {58 vars in 1275 bytes} [Fri Oct 28 11:16:47 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-interface => generated 285 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:16:48.188228 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-6bcd0d4f-e8af-416c-8369-8c75445fa83b req-d59583f7-7c2d-4b66-98f3-5fb9e17e3e09 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cd5eee5e-9d7f-40b0-abd1-0393d82f90ad", "tag": "a924d329-0af3-4eae-b277-6e7253359d9f"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:16:48.198644 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6bcd0d4f-e8af-416c-8369-8c75445fa83b req-d59583f7-7c2d-4b66-98f3-5fb9e17e3e09 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:16:48.199116 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6bcd0d4f-e8af-416c-8369-8c75445fa83b req-d59583f7-7c2d-4b66-98f3-5fb9e17e3e09 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:16:48.199541 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-6bcd0d4f-e8af-416c-8369-8c75445fa83b req-d59583f7-7c2d-4b66-98f3-5fb9e17e3e09 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:16:48.211066 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-6bcd0d4f-e8af-416c-8369-8c75445fa83b req-d59583f7-7c2d-4b66-98f3-5fb9e17e3e09 service nova] Creating event network-changed:a924d329-0af3-4eae-b277-6e7253359d9f for instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad on hv-cinder-zed-3 Oct 28 11:16:48.215746 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-6bcd0d4f-e8af-416c-8369-8c75445fa83b req-d59583f7-7c2d-4b66-98f3-5fb9e17e3e09 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031173 Oct 28 11:16:48.216336 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 577/1140] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:16:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:17:09.445194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=13 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:09.445935 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=39 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:09.450077 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:09.450571 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:09.457790 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:09.458297 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:09.458735 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:09.459420 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:09.459869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:09.460293 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:09.538942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lazy-loading 'services' on Instance uuid 3ad6871e-a8cc-443d-ab6c-ef650207f988 {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:17:09.607617 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4ebc8d33-0d80-4c65-8e46-29d07ec09171 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "DELETE /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988/os-volume_attachments/51f24d1f-2969-42a4-a002-cfded573d8d9" status: 202 len: 0 microversion: 2.1 time: 0.163583 Oct 28 11:17:09.608199 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 564/1141] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:17:09 2022] DELETE /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988/os-volume_attachments/51f24d1f-2969-42a4-a002-cfded573d8d9 => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:17:10.734283 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=13 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:10.735022 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=34,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:10.739342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:10.740056 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:10.747567 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:10.748074 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:10.748503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:10.749178 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:10.749658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:10.750106 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:10.776134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:17:10.850619 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b7ce567e-a4eb-43dd-b36d-5932d16cfa24 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "DELETE /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 204 len: 0 microversion: 2.1 time: 0.117559 Oct 28 11:17:10.851148 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 578/1142] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:17:10 2022] DELETE /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:17:10.860742 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:10.861444 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:10.867863 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:10.868347 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:10.868786 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:10.869744 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:10.870231 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:10.870657 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:10.999202 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:11.007501 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-c71f9b3f-ccf8-4cc1-bdff-98c73e727437 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1514 microversion: 2.1 time: 0.151450 Oct 28 11:17:11.008075 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 565/1143] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:10 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1514 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:12.030524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:12.031215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:12.038216 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:12.038692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:12.039120 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:12.039798 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:12.040245 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:12.040661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:12.169795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:12.177955 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-8364e62b-24da-4626-8018-c5e932cddbb2 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1514 microversion: 2.1 time: 0.151777 Oct 28 11:17:12.178531 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 579/1144] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:12 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1514 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:13.200713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:13.201411 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:13.207771 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:13.208249 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:13.208880 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:13.209629 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:13.210114 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:13.210540 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:13.284627 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:13.292496 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7f039086-2924-4493-9056-6c97fbf9c5e1 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 200 len: 1472 microversion: 2.1 time: 0.095667 Oct 28 11:17:13.298706 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 566/1145] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:13 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 1472 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:13.316131 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-fbf928cb-6a73-4e8b-b7ee-0edbdfe99538 req-13a98702-70fd-463b-a89f-4b7c38cf480b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3ad6871e-a8cc-443d-ab6c-ef650207f988", "name": "network-vif-deleted", "tag": "ac892418-282c-4436-8152-650c1baa89c6"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:17:13.325342 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-fbf928cb-6a73-4e8b-b7ee-0edbdfe99538 req-13a98702-70fd-463b-a89f-4b7c38cf480b service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:13.325810 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-fbf928cb-6a73-4e8b-b7ee-0edbdfe99538 req-13a98702-70fd-463b-a89f-4b7c38cf480b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:13.326260 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-fbf928cb-6a73-4e8b-b7ee-0edbdfe99538 req-13a98702-70fd-463b-a89f-4b7c38cf480b service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:13.337601 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-fbf928cb-6a73-4e8b-b7ee-0edbdfe99538 req-13a98702-70fd-463b-a89f-4b7c38cf480b service nova] Creating event network-vif-deleted:ac892418-282c-4436-8152-650c1baa89c6 for instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 on hv-cinder-zed-3 Oct 28 11:17:13.342645 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-fbf928cb-6a73-4e8b-b7ee-0edbdfe99538 req-13a98702-70fd-463b-a89f-4b7c38cf480b service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101746 Oct 28 11:17:13.343234 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 580/1146] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:17:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:17:14.317366 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:14.318056 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 3ad6871e-a8cc-443d-ab6c-ef650207f988] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:14.324443 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:14.324925 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:14.325362 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:14.326084 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:14.326542 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:14.326965 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:14.339931 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] HTTP exception thrown: Instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 could not be found. Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Request method failure captured: Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 HTTP/1.1 Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7n0BGR-BO9HsT3uBP2xc5WEaU3j0GtR7l4fB13jfzxbpPlk_V0oCwsDcyW7xzHE4CMrMHi1NHgIIxO-QpLCv2QoZdp6VbC2gxEnOSy4D80KlfCj_vulgdzyz0L9u_56wUhIQumLKQFkrUfLx49E2ZANRWBOwpDl0mnQtek2YUz2HU0 Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 33af503fc7e9410885f704ffbfcb900b Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-VolumesSnapshotTestJSON-440258102 Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:17:14.341572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-VolumesSnapshotTestJSON-440258102 Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 33af503fc7e9410885f704ffbfcb900b Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-440258102 Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-VolumesSnapshotTestJSON-440258102-project Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: c689e56cfc3f43e187c8804fd8577cb7 Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-VolumesSnapshotTestJSON-440258102-project Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 could not be found. Oct 28 11:17:14.343425 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '3ad6871e-a8cc-443d-ab6c-ef650207f988'} Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 could not be found. Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:17:14.344649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 could not be found. Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 could not be found. Oct 28 11:17:14.347002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:17:14.348564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Returning 404 to user: Instance 3ad6871e-a8cc-443d-ab6c-ef650207f988 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:17:14.348564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-abd33118-206b-47c3-96db-80ba25cbf582 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988" status: 404 len: 111 microversion: 2.1 time: 0.030203 Oct 28 11:17:14.348564 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 567/1147] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:14 2022] GET /compute/v2.1/servers/3ad6871e-a8cc-443d-ab6c-ef650207f988 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:17:24.159365 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:24.160753 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=22,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:24.165207 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1749515355", "imageRef": "200554c1-4f37-4421-954b-8091133f8e11", "flavorRef": "452", "networks": [{"uuid": "4dbdb288-c7d5-4203-bd75-39f48212d6ac"}]}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:17:24.219640 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.network.neutron [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] validate_networks() for [('4dbdb288-c7d5-4203-bd75-39f48212d6ac', None, None, None, None, None)] {{(pid=120543) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 28 11:17:24.555118 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Flavor limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 28 11:17:24.555542 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Image limits 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 28 11:17:24.556009 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Flavor pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 28 11:17:24.556418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Image pref 0:0:0 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 28 11:17:24.556825 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.virt.hardware [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120543) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 28 11:17:24.565544 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:24.566052 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:24.566484 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:24.567208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:24.567661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:24.568082 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:24.585093 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Getting quotas for project 33af503fc7e9410885f704ffbfcb900b. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 28 11:17:24.589030 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.quota [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Getting quotas for user c689e56cfc3f43e187c8804fd8577cb7 and project 33af503fc7e9410885f704ffbfcb900b. Resources: {'cores', 'instances', 'ram'} {{(pid=120543) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 28 11:17:24.600502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Going to run 1 instances... {{(pid=120543) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 28 11:17:24.602811 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='200554c1-4f37-4421-954b-8091133f8e11',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120543) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 28 11:17:24.648862 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-deda9849-1991-4b16-84ec-aaa990471675 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.491003 Oct 28 11:17:24.649536 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 581/1148] 10.222.0.46 () {60 vars in 1147 bytes} [Fri Oct 28 11:17:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 492 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 28 11:17:24.661499 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-0b186651-d9ad-433e-ab64-827510b52188 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:24.662145 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-0b186651-d9ad-433e-ab64-827510b52188 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:24.664770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=8 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:24.665556 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:24.721039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-0b186651-d9ad-433e-ab64-827510b52188 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:24.728536 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-0b186651-d9ad-433e-ab64-827510b52188 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1219 microversion: 2.1 time: 0.071425 Oct 28 11:17:24.729105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 568/1149] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:24 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1219 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:25.752165 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:25.753986 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:25.761580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:25.762328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:25.762745 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:25.763511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:25.763961 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:25.764451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:25.849596 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:25.857852 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-78706961-1441-4f75-a194-139780607e68 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1289 microversion: 2.1 time: 0.110278 Oct 28 11:17:25.858454 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 582/1150] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:25 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1289 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:26.881310 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:26.881920 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:26.889173 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:26.889653 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:26.890105 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:26.890792 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:26.891251 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:26.891669 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:27.000610 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-31f114c5-507a-425d-97bf-b54ffdc71226 req-6ee73986-4325-430a-9c93-afe862f570d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "92d01e53-f4c7-49ff-8675-d16f70714fb5", "tag": "8a60b0e9-7c14-4d94-9e0d-0502eb26ffd1"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:17:27.012861 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-31f114c5-507a-425d-97bf-b54ffdc71226 req-6ee73986-4325-430a-9c93-afe862f570d3 service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:27.022916 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-31f114c5-507a-425d-97bf-b54ffdc71226 req-6ee73986-4325-430a-9c93-afe862f570d3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:27.023800 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-31f114c5-507a-425d-97bf-b54ffdc71226 req-6ee73986-4325-430a-9c93-afe862f570d3 service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:27.031123 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:27.037558 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-31f114c5-507a-425d-97bf-b54ffdc71226 req-6ee73986-4325-430a-9c93-afe862f570d3 service nova] Creating event network-changed:8a60b0e9-7c14-4d94-9e0d-0502eb26ffd1 for instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 on hv-cinder-zed-3 Oct 28 11:17:27.040479 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a7081fa0-d01b-4cc8-9de7-6f44396c4981 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.163584 Oct 28 11:17:27.041069 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 569/1151] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:26 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:27.042945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-31f114c5-507a-425d-97bf-b54ffdc71226 req-6ee73986-4325-430a-9c93-afe862f570d3 service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046679 Oct 28 11:17:27.043975 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 583/1152] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:17:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:17:28.063812 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:28.064407 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:28.071254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:28.071765 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:28.072201 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:28.072889 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:28.073346 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:28.073770 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:28.204178 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:28.211807 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-7d3c4173-0829-4eb8-a374-f02a44b85a47 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.151900 Oct 28 11:17:28.212372 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 570/1153] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:28 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:29.234543 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:29.235236 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:29.241857 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:29.242373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:29.242802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:29.243478 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:29.243926 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:29.244597 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:29.373892 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:29.381453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-eeada09b-5a29-47b3-ab54-77b6733b4691 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.151580 Oct 28 11:17:29.382031 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 584/1154] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:29 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:30.404142 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:30.404711 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:30.411111 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:30.411576 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:30.412004 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:30.412688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:30.413137 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:30.413555 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:30.539714 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:30.547374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-e585783b-1a56-4da1-ac1f-4499b52fda1c tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.147132 Oct 28 11:17:30.547987 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 571/1155] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:30 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:31.571145 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:31.571854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:31.578269 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:31.578752 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:31.579176 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:31.579854 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:31.580305 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:31.580727 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:31.711581 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:31.719647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-f620371e-1fde-40bc-abde-62cf5ab6150a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.152370 Oct 28 11:17:31.720217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 585/1156] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:31 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:32.742835 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:32.743384 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:32.749737 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:32.750242 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:32.750674 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:32.751353 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:32.751802 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:32.752222 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:32.876943 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:32.885149 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-2e4abec6-c558-480e-9d0f-8b32eca260c9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.146273 Oct 28 11:17:32.885716 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 572/1157] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:32 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:33.408496 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [req-90a86518-ad46-4f72-be12-ca707c61a1da req-3a279bb6-1e25-4b18-9b4e-c1b7f4fda84c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92d01e53-f4c7-49ff-8675-d16f70714fb5", "name": "network-vif-plugged", "status": "completed", "tag": "8a60b0e9-7c14-4d94-9e0d-0502eb26ffd1"}]} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:17:33.420721 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-90a86518-ad46-4f72-be12-ca707c61a1da req-3a279bb6-1e25-4b18-9b4e-c1b7f4fda84c service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:33.421197 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-90a86518-ad46-4f72-be12-ca707c61a1da req-3a279bb6-1e25-4b18-9b4e-c1b7f4fda84c service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:33.421622 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [req-90a86518-ad46-4f72-be12-ca707c61a1da req-3a279bb6-1e25-4b18-9b4e-c1b7f4fda84c service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:33.432945 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.compute.server_external_events [req-90a86518-ad46-4f72-be12-ca707c61a1da req-3a279bb6-1e25-4b18-9b4e-c1b7f4fda84c service nova] Creating event network-vif-plugged:8a60b0e9-7c14-4d94-9e0d-0502eb26ffd1 for instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 on hv-cinder-zed-3 Oct 28 11:17:33.437851 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [req-90a86518-ad46-4f72-be12-ca707c61a1da req-3a279bb6-1e25-4b18-9b4e-c1b7f4fda84c service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090540 Oct 28 11:17:33.438451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 586/1158] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:17:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 28 11:17:33.908171 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:33.908695 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:33.915713 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:33.916225 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:33.916684 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:33.917406 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:33.917858 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:33.918304 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:34.045958 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:34.054129 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-563e48ff-b3b3-45cf-bbd2-efb056833421 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.149957 Oct 28 11:17:34.054698 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 573/1159] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:33 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:35.074347 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:35.075115 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell0:SELECT=6 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:35.079187 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:35.081884 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:35.087440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:35.087914 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:35.088351 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:35.089038 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:35.089493 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:35.089918 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:35.219733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:35.228006 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d391efca-988b-4213-b327-ae094f74fc26 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1319 microversion: 2.1 time: 0.155081 Oct 28 11:17:35.228573 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 587/1160] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:35 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:36.250855 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:36.251395 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:36.253710 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell0:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:36.259282 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:36.259753 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:36.260175 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:36.260878 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:36.261323 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:36.261734 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:36.389236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:36.397489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-20f6c06f-9ee8-46f4-a427-5f36fc0a1dba tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1508 microversion: 2.1 time: 0.150548 Oct 28 11:17:36.398089 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 574/1161] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:36 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1508 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:17:36.419660 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5a353825-b5e2-4037-b1b6-28f06e455945", "device": "/dev/sdb"}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:17:36.422140 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:36.429171 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:36.429647 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:36.430113 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:36.430795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:36.431242 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:36.431658 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:36.666246 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-70dcf560-c086-4428-839a-1f0f658dba88 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "POST /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.250420 Oct 28 11:17:36.666880 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 588/1162] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:17:36 2022] POST /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5/os-volume_attachments => generated 194 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:17:58.058580 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:58.059096 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:58.061472 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=16 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:58.062189 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=36 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:58.067868 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:58.068341 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:58.068773 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:58.069454 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:58.069901 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:58.070355 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:58.150001 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.objects.instance [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lazy-loading 'services' on Instance uuid 92d01e53-f4c7-49ff-8675-d16f70714fb5 {{(pid=120544) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:17:58.219065 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-a4ca8aba-352f-4f3b-accb-b7b802e54397 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "DELETE /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5/os-volume_attachments/5a353825-b5e2-4037-b1b6-28f06e455945" status: 202 len: 0 microversion: 2.1 time: 0.164235 Oct 28 11:17:58.219779 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 575/1163] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:17:58 2022] DELETE /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5/os-volume_attachments/5a353825-b5e2-4037-b1b6-28f06e455945 => generated 0 bytes in 166 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:17:59.340307 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=3 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:59.341083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=41,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:17:59.345210 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:59.345787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:59.354453 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:59.354927 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:59.355361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:59.356040 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:59.356503 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:59.356921 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:59.382682 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Going to try to terminate instance {{(pid=120543) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:17:59.452845 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-9b2fd027-cceb-4d34-9863-84f57193d00a tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "DELETE /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 204 len: 0 microversion: 2.1 time: 0.113801 Oct 28 11:17:59.453444 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 589/1164] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:17:59 2022] DELETE /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 28 11:17:59.462700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:17:59.463399 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:17:59.469513 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:59.470002 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:59.470498 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:59.471177 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:17:59.471621 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:17:59.472035 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:17:59.601427 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:17:59.609136 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d245c9b4-536d-413a-87c9-4c99279e5017 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1514 microversion: 2.1 time: 0.151098 Oct 28 11:17:59.609693 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 576/1165] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:17:59 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1514 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:18:00.633695 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:00.634418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:00.640999 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:00.641487 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:00.641925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:00.642653 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:00.643110 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:00.643538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:00.770439 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:18:00.778083 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-a9e7b0ac-1b6c-4887-a712-d46c5be62709 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1514 microversion: 2.1 time: 0.149161 Oct 28 11:18:00.778659 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 590/1166] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:00 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1514 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:18:01.674494 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-661e58de-e9ed-4ec9-a5ae-a884eeb31fae req-cb2176cb-751b-4d11-966c-2d25dcf1dbef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92d01e53-f4c7-49ff-8675-d16f70714fb5", "name": "network-vif-deleted", "tag": "8a60b0e9-7c14-4d94-9e0d-0502eb26ffd1"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:18:01.685217 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-661e58de-e9ed-4ec9-a5ae-a884eeb31fae req-cb2176cb-751b-4d11-966c-2d25dcf1dbef service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:01.685687 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-661e58de-e9ed-4ec9-a5ae-a884eeb31fae req-cb2176cb-751b-4d11-966c-2d25dcf1dbef service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:01.686143 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-661e58de-e9ed-4ec9-a5ae-a884eeb31fae req-cb2176cb-751b-4d11-966c-2d25dcf1dbef service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:01.697637 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-661e58de-e9ed-4ec9-a5ae-a884eeb31fae req-cb2176cb-751b-4d11-966c-2d25dcf1dbef service nova] Creating event network-vif-deleted:8a60b0e9-7c14-4d94-9e0d-0502eb26ffd1 for instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 on hv-cinder-zed-3 Oct 28 11:18:01.702493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-661e58de-e9ed-4ec9-a5ae-a884eeb31fae req-cb2176cb-751b-4d11-966c-2d25dcf1dbef service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031624 Oct 28 11:18:01.703082 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 577/1167] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:18:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:18:01.801054 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:01.801666 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:01.808299 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:01.808771 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:01.809217 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:01.809886 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:01.810361 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:01.810781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:01.843373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:01.843866 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:01.844277 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:01.844808 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lazy-loading 'fault' on Instance uuid 92d01e53-f4c7-49ff-8675-d16f70714fb5 {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:18:01.885792 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c689e56cfc3f43e187c8804fd8577cb7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '33af503fc7e9410885f704ffbfcb900b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:18:01.893461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-469c4573-3e19-4e57-9ed7-cf7379e2e5a9 tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 200 len: 1309 microversion: 2.1 time: 0.096278 Oct 28 11:18:01.894045 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 591/1168] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:01 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 1309 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:18:02.917970 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:02.918703 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] [instance: 92d01e53-f4c7-49ff-8675-d16f70714fb5] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:02.924831 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:02.925308 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:02.925744 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:02.926468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:02.926925 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:02.927354 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:02.940284 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] HTTP exception thrown: Instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 could not be found. Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Request method failure captured: Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 HTTP/1.1 Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7n0BGR-BO9HsT3uBP2xc5WEaU3j0GtR7l4fB13jfzxbpPlk_V0oCwsDcyW7xzHE4CMrMHi1NHgIIxO-QpLCv2QoZdp6VbC2gxEnOSy4D80KlfCj_vulgdzyz0L9u_56wUhIQumLKQFkrUfLx49E2ZANRWBOwpDl0mnQtek2YUz2HU0 Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 33af503fc7e9410885f704ffbfcb900b Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-VolumesSnapshotTestJSON-440258102 Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:02.941869 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/33af503fc7e9410885f704ffbfcb900b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-VolumesSnapshotTestJSON-440258102 Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 33af503fc7e9410885f704ffbfcb900b Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-440258102 Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-VolumesSnapshotTestJSON-440258102-project Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: c689e56cfc3f43e187c8804fd8577cb7 Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-VolumesSnapshotTestJSON-440258102-project Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 could not be found. Oct 28 11:18:02.943726 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '92d01e53-f4c7-49ff-8675-d16f70714fb5'} Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 could not be found. Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:02.944699 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 could not be found. Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 could not be found. Oct 28 11:18:02.946374 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:02.947909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] Returning 404 to user: Instance 92d01e53-f4c7-49ff-8675-d16f70714fb5 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:02.947909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-49023a14-0c51-424c-aec9-d3254a5d586f tempest-VolumesSnapshotTestJSON-440258102 tempest-VolumesSnapshotTestJSON-440258102-project] 10.222.0.46 "GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5" status: 404 len: 111 microversion: 2.1 time: 0.030154 Oct 28 11:18:02.947909 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 578/1169] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:02 2022] GET /compute/v2.1/servers/92d01e53-f4c7-49ff-8675-d16f70714fb5 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:30.347787 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:30.348294 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:30.351300 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:18:30.351943 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:18:30.359215 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:30.359733 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:30.360167 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:30.360875 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:30.361301 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:30.361727 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:30.385663 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c763320d-428d-440f-a76b-9441938fc8cb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/ips" status: 200 len: 150 microversion: 2.1 time: 0.041773 Oct 28 11:18:30.386264 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 592/1170] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:18:30 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:18:34.865733 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b4474211-d0ff-48ce-a2d9-15d788563a9a"}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:18:34.868089 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:34.870184 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=8 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:18:34.871090 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:18:34.880123 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:34.880809 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:34.881240 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:34.881917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:34.882386 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:34.882806 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:38.075588 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-4bb7637b-ef3d-425f-9e98-6fbd330ab94e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "POST /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.213796 Oct 28 11:18:38.076449 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 579/1171] 10.222.0.46 () {60 vars in 1323 bytes} [Fri Oct 28 11:18:34 2022] POST /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments => generated 194 bytes in 3215 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:18:45.717109 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:45.717775 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:45.719869 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_api:SELECT=2 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:18:45.720511 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG dbcounter [-] [120543] Writing DB stats nova_cell1:SELECT=4 {{(pid=120543) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:18:45.726925 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:45.727403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:45.727837 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:45.728530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:45.729036 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:45.729461 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:45.810373 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.objects.instance [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lazy-loading 'services' on Instance uuid cd5eee5e-9d7f-40b0-abd1-0393d82f90ad {{(pid=120543) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 28 11:18:45.879444 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-d7788d03-9460-4b59-9159-ef139621ba37 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments/b4474211-d0ff-48ce-a2d9-15d788563a9a" status: 202 len: 0 microversion: 2.1 time: 0.166223 Oct 28 11:18:45.880098 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 593/1172] 10.222.0.46 () {58 vars in 1416 bytes} [Fri Oct 28 11:18:45 2022] DELETE /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments/b4474211-d0ff-48ce-a2d9-15d788563a9a => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:18:45.888937 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:45.891194 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:45.893669 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG dbcounter [-] [120544] Writing DB stats nova_api:SELECT=2 {{(pid=120544) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 11:18:45.898692 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:45.899165 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:45.899607 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:45.900288 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:45.900730 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:45.901167 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:45.932648 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-14ba2b08-b829-4bc3-9122-a35b55c80da9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047974 Oct 28 11:18:45.933143 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 580/1173] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:18:45 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:18:46.946388 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:46.948692 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:46.954766 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:46.955239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:46.955664 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:46.956341 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:46.956786 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:46.957205 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:46.987781 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e002f98b-118b-4b13-bcbb-f59c4670fc2e tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046468 Oct 28 11:18:46.988328 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 594/1174] 10.222.0.46 () {58 vars in 1302 bytes} [Fri Oct 28 11:18:46 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 28 11:18:47.049900 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:47.050409 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:47.056634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:47.057113 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:47.057544 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:47.058246 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:47.058694 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:47.059112 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:47.085365 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Going to try to terminate instance {{(pid=120544) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 28 11:18:47.158366 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-1db84283-44fc-48e6-9930-9376d8d2e650 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 204 len: 0 microversion: 2.1 time: 0.111930 Oct 28 11:18:47.159083 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 581/1175] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:18:47 2022] DELETE /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 28 11:18:47.168185 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:47.168789 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:47.174661 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:47.175134 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:47.175561 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:47.176232 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:47.176674 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:47.177087 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:47.305418 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:18:47.313075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-2969cf98-8c79-4a33-a288-f5b9a2f0c09a tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1679 microversion: 2.1 time: 0.148877 Oct 28 11:18:47.313638 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 595/1176] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:47 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1679 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:18:48.337780 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:48.338566 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:48.345098 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:48.345581 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:48.346042 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:48.346746 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:48.347211 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:48.347634 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:48.476847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.policy [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120544) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:18:48.484485 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-d467c5d8-cdc5-4172-b7bd-416827927055 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1679 microversion: 2.1 time: 0.152218 Oct 28 11:18:48.485052 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 582/1177] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:48 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:18:49.508591 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:49.509202 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:49.515595 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:49.516070 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:49.516501 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:49.517195 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:49.517643 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:49.518091 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:49.607550 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.policy [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ff4b7731f96643d0a24e63d31f5f6f00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '68345feb30fe454083122ecaf73ef747', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120543) authorize /opt/stack/nova/nova/policy.py:199}} Oct 28 11:18:49.615965 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-afe479cb-a39d-44f4-825b-b4036053aa45 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 200 len: 1612 microversion: 2.1 time: 0.111073 Oct 28 11:18:49.615965 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 596/1178] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:49 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 1612 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 28 11:18:49.670039 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [req-ff713b65-81b2-4aee-a369-af827bf8c8a0 req-19e9879a-8188-4fc4-b0b4-90833741649e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cd5eee5e-9d7f-40b0-abd1-0393d82f90ad", "name": "network-vif-deleted", "tag": "a924d329-0af3-4eae-b277-6e7253359d9f"}]} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:18:49.680759 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ff713b65-81b2-4aee-a369-af827bf8c8a0 req-19e9879a-8188-4fc4-b0b4-90833741649e service nova] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:49.681234 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ff713b65-81b2-4aee-a369-af827bf8c8a0 req-19e9879a-8188-4fc4-b0b4-90833741649e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:49.681649 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [req-ff713b65-81b2-4aee-a369-af827bf8c8a0 req-19e9879a-8188-4fc4-b0b4-90833741649e service nova] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:49.693134 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.compute.server_external_events [req-ff713b65-81b2-4aee-a369-af827bf8c8a0 req-19e9879a-8188-4fc4-b0b4-90833741649e service nova] Creating event network-vif-deleted:a924d329-0af3-4eae-b277-6e7253359d9f for instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad on hv-cinder-zed-3 Oct 28 11:18:49.698725 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [req-ff713b65-81b2-4aee-a369-af827bf8c8a0 req-19e9879a-8188-4fc4-b0b4-90833741649e service nova] 10.222.0.46 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032333 Oct 28 11:18:49.699453 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 583/1179] 10.222.0.46 () {64 vars in 1317 bytes} [Fri Oct 28 11:18:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 28 11:18:50.639267 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:50.639883 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:50.646843 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:50.647255 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:50.647713 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:50.648403 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:50.648841 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:50.649538 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:50.663201 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad HTTP/1.1 Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:50.664893 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:50.666502 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'cd5eee5e-9d7f-40b0-abd1-0393d82f90ad'} Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:50.667519 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:50.671481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:50.674795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:50.674795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-dc5987d5-1148-4ed9-9546-dc055acfa342 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 404 len: 111 microversion: 2.1 time: 0.031015 Oct 28 11:18:50.674795 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 597/1180] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:50 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:50.678006 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-009c085e-0c9a-483b-ac2e-1e1d2ffe8586 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '.version_select at 0x7f42fa87bee0>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:50.699832 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-009c085e-0c9a-483b-ac2e-1e1d2ffe8586 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-503492479" status: 202 len: 0 microversion: 2.1 time: 0.027190 Oct 28 11:18:50.700467 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 584/1181] 10.222.0.46 () {58 vars in 1218 bytes} [Fri Oct 28 11:18:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-503492479 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 28 11:18:52.451481 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:52.451890 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:52.458315 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:52.458792 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:52.459221 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:52.459930 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:52.461043 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:52.461464 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:52.474669 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad HTTP/1.1 Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:52.476239 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.477894 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'cd5eee5e-9d7f-40b0-abd1-0393d82f90ad'} Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:52.478897 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.480451 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.482172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:52.482172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-60c51088-05b6-418e-83b6-82e120a5b361 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 404 len: 111 microversion: 2.1 time: 0.030004 Oct 28 11:18:52.482172 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 598/1182] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:18:52 2022] DELETE /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:52.486423 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:52.487151 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: cd5eee5e-9d7f-40b0-abd1-0393d82f90ad] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:52.493766 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:52.494818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:52.496178 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:52.496398 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:52.496852 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:52.497275 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:52.510595 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad HTTP/1.1 Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:52.512182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.513847 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'cd5eee5e-9d7f-40b0-abd1-0393d82f90ad'} Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:52.514829 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. Oct 28 11:18:52.516266 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:52.517468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance cd5eee5e-9d7f-40b0-abd1-0393d82f90ad could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:52.517468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-dd628e31-2af3-4393-ab97-5a4e45797ac9 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad" status: 404 len: 111 microversion: 2.1 time: 0.031416 Oct 28 11:18:52.517468 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 585/1183] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:52 2022] GET /compute/v2.1/servers/cd5eee5e-9d7f-40b0-abd1-0393d82f90ad => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:53.648812 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:53.649314 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:53.655697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.656180 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.656614 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.657305 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.657755 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.658208 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.671338 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d HTTP/1.1 Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:53.672865 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.674603 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d'} Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:53.675744 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.677407 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.679616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:53.679616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e562e5be-ba0f-4579-a6bc-9e2e65dd1dae tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 404 len: 111 microversion: 2.1 time: 0.029302 Oct 28 11:18:53.679616 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 599/1184] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:18:53 2022] DELETE /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:53.683917 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:53.684527 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:53.690764 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.691235 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.691659 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.692333 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.692814 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.693236 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.706639 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d HTTP/1.1 Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:53.708110 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.709799 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d'} Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:53.710981 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. Oct 28 11:18:53.712572 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.714254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:53.714254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-fd45673a-ecfa-4848-a463-2573d73840bb tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d" status: 404 len: 111 microversion: 2.1 time: 0.029593 Oct 28 11:18:53.714254 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 586/1185] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:53 2022] GET /compute/v2.1/servers/b6cdc1e6-ff16-4c8a-8af0-43b1adcbe16d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:53.717782 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:53.718294 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:53.724117 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.724583 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.725012 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.725802 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.726272 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.726689 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.739657 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d HTTP/1.1 Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:53.741703 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.743432 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'ddc0c4cb-9db0-435e-b0bf-64320ce1c24d'} Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:53.744524 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.746530 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.748124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:53.748124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-72274992-cdc6-45dc-9e9d-547f5eaa339c tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 404 len: 111 microversion: 2.1 time: 0.029037 Oct 28 11:18:53.748124 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 600/1186] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:18:53 2022] DELETE /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 11:18:53.751489 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:53.752091 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: ddc0c4cb-9db0-435e-b0bf-64320ce1c24d] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:53.758486 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.758957 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.759381 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.760051 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:53.760493 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:53.760915 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:53.773800 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d HTTP/1.1 Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:53.775319 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.777022 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'ddc0c4cb-9db0-435e-b0bf-64320ce1c24d'} Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:53.778155 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. Oct 28 11:18:53.779700 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:53.781127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance ddc0c4cb-9db0-435e-b0bf-64320ce1c24d could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:53.781127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f664a201-86fb-4706-a369-981ba4bf95a1 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d" status: 404 len: 111 microversion: 2.1 time: 0.028906 Oct 28 11:18:53.781127 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 587/1187] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:53 2022] GET /compute/v2.1/servers/ddc0c4cb-9db0-435e-b0bf-64320ce1c24d => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 11:18:54.911842 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:54.912337 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.compute.api [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120543) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:54.919320 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:54.919797 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:54.920226 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:54.920909 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:54.921355 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:54.921775 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG oslo_concurrency.lockutils [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120543) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:54.935569 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 HTTP/1.1 Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:54.937079 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: > Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.938824 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': '952a161f-410d-4266-b236-14823a631423'} Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:54.939946 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.941629 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.943191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:54.943191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-3ea98fd7-6484-408f-a737-7e97c1057e47 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "DELETE /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 404 len: 111 microversion: 2.1 time: 0.030520 Oct 28 11:18:54.943191 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 601/1188] 10.222.0.46 () {58 vars in 1239 bytes} [Fri Oct 28 11:18:54 2022] DELETE /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:54.946573 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Calling method '>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:54.947182 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.compute.api [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] [instance: 952a161f-410d-4266-b236-14823a631423] Fetching instance by UUID {{(pid=120544) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 28 11:18:54.953529 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:54.954029 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:54.954463 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:54.955141 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Acquiring lock "38276e5b-4682-4472-aad8-debabc8dd457" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 28 11:18:54.955594 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 28 11:18:54.956010 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG oslo_concurrency.lockutils [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Lock "38276e5b-4682-4472-aad8-debabc8dd457" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 28 11:18:54.968888 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] HTTP exception thrown: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Request method failure captured: Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 HTTP/1.1 Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7jgH5AXDxV8cRdeLXBq3IgB6xkTrQtZwi-KmcdusFVKibf_2oVT5VQZ9dl_xolWWZbVHmgbRW6FGQzrC1PBdeO3YFdLnXhiCLWCiDRstHyXD01VDMSyc8RbGvdhq6bFQnM-Q1fKqxIxn9W4vjcsEt-Deb3Iouhb0qwBWJ3FfQ6kzdQ Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:54.970389 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/68345feb30fe454083122ecaf73ef747"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_68345feb30fe454083122ecaf73ef747"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/68345feb30fe454083122ecaf73ef747"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 68345feb30fe454083122ecaf73ef747 Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1425575815 Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: ff4b7731f96643d0a24e63d31f5f6f00 Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-AttachVolumeNegativeTest-1425575815-project Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: > Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.972058 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '952a161f-410d-4266-b236-14823a631423'} Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:54.973060 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. Oct 28 11:18:54.974521 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:54.976380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] Returning 404 to user: Instance 952a161f-410d-4266-b236-14823a631423 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:54.976380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-22411207-8e78-47ef-bbb7-d588fd1ac8e8 tempest-AttachVolumeNegativeTest-1425575815 tempest-AttachVolumeNegativeTest-1425575815-project] 10.222.0.46 "GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423" status: 404 len: 111 microversion: 2.1 time: 0.028931 Oct 28 11:18:54.976380 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 588/1189] 10.222.0.46 () {58 vars in 1236 bytes} [Fri Oct 28 11:18:54 2022] GET /compute/v2.1/servers/952a161f-410d-4266-b236-14823a631423 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 28 11:18:59.406026 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b34c9eb4-0942-419b-95df-c9f696ae0d3e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Action: 'create', calling method: .version_select at 0x7f42fa615940>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1931418799", "metadata": {"Type": "work"}}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b34c9eb4-0942-419b-95df-c9f696ae0d3e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Request method failure captured: Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 122 Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7qjfpujfQAOPHzbbsmU1QeXfqjjL_q19_F1bWkb_jLLPq-iPHwWd2JtykgCQjpYJ-fViV8TyduV-LHG7JjjDsbG3b8NVpLlHoMU_RUx-4N4lFCzrzX-D0DjumqeRiK7C_BTO0cv2LqNIkRsRVOWoMgYjxpJUxaUxIF5ySnElZWvOvA Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:59.427584 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/72e52c52a2524fc4a69040c818109867"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_72e52c52a2524fc4a69040c818109867"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: c25289bd54b24b7dabb300a53ccb4959 Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1931418799", "metadata": {"Type": "work"}}} Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:18:59.429362 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1931418799', 'metadata': {'Type': 'work'}}}} Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise error Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi '#$%' Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:59.430348 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 28 11:18:59.432092 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 28 11:18:59.433447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 28 11:18:59.433447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.433447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-b34c9eb4-0942-419b-95df-c9f696ae0d3e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:59.433447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-b34c9eb4-0942-419b-95df-c9f696ae0d3e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.133703 Oct 28 11:18:59.433447 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 602/1190] 10.222.0.46 () {60 vars in 1156 bytes} [Fri Oct 28 11:18:59 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 135 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:18:59.438518 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f21d09db-c247-4601-aa8d-009de6ecec4d tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Action: 'create', calling method: .version_select at 0x7f42fb457550>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-779536236", "metadata": {"Type": "work"}}} {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f21d09db-c247-4601-aa8d-009de6ecec4d tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Request method failure captured: Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 119 Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7qjfpujfQAOPHzbbsmU1QeXfqjjL_q19_F1bWkb_jLLPq-iPHwWd2JtykgCQjpYJ-fViV8TyduV-LHG7JjjDsbG3b8NVpLlHoMU_RUx-4N4lFCzrzX-D0DjumqeRiK7C_BTO0cv2LqNIkRsRVOWoMgYjxpJUxaUxIF5ySnElZWvOvA Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:59.442944 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/72e52c52a2524fc4a69040c818109867"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_72e52c52a2524fc4a69040c818109867"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: c25289bd54b24b7dabb300a53ccb4959 Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-779536236", "metadata": {"Type": "work"}}} Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: Oct 28 11:18:59.445188 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-779536236', 'metadata': {'Type': 'work'}}}} Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise error Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi '0' Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:59.446265 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 28 11:18:59.447945 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 28 11:18:59.449375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 28 11:18:59.449375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.449375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f21d09db-c247-4601-aa8d-009de6ecec4d tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:59.449375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f21d09db-c247-4601-aa8d-009de6ecec4d tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.009550 Oct 28 11:18:59.449375 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 589/1191] 10.222.0.46 () {60 vars in 1156 bytes} [Fri Oct 28 11:18:59 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:18:59.453369 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-364482d8-9da1-4571-83d7-02eb95aa4f34 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Action: 'create', calling method: .version_select at 0x7f42fa40cdc0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1069926880", "metadata": {"Type": "work"}}} {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-364482d8-9da1-4571-83d7-02eb95aa4f34 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Request method failure captured: Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 119 Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7qjfpujfQAOPHzbbsmU1QeXfqjjL_q19_F1bWkb_jLLPq-iPHwWd2JtykgCQjpYJ-fViV8TyduV-LHG7JjjDsbG3b8NVpLlHoMU_RUx-4N4lFCzrzX-D0DjumqeRiK7C_BTO0cv2LqNIkRsRVOWoMgYjxpJUxaUxIF5ySnElZWvOvA Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:59.457827 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/72e52c52a2524fc4a69040c818109867"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_72e52c52a2524fc4a69040c818109867"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: c25289bd54b24b7dabb300a53ccb4959 Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1069926880", "metadata": {"Type": "work"}}} Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:18:59.459560 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1069926880', 'metadata': {'Type': 'work'}}}} Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise error Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi '' Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:59.461146 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 28 11:18:59.462809 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 28 11:18:59.464181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 28 11:18:59.464181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.464181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-364482d8-9da1-4571-83d7-02eb95aa4f34 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:59.464181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-364482d8-9da1-4571-83d7-02eb95aa4f34 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.009484 Oct 28 11:18:59.464181 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 603/1192] 10.222.0.46 () {60 vars in 1156 bytes} [Fri Oct 28 11:18:59 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 28 11:18:59.468942 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f81b2f52-4947-4601-8703-5750003a72aa tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Calling method '.version_select at 0x7f42fa788f70>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:59.497902 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-f81b2f52-4947-4601-8703-5750003a72aa tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] HTTP exception thrown: Volume tempest-invalid-2040715802 could not be found. Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f81b2f52-4947-4601-8703-5750003a72aa tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Request method failure captured: Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-2040715802 HTTP/1.1 Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7qjfpujfQAOPHzbbsmU1QeXfqjjL_q19_F1bWkb_jLLPq-iPHwWd2JtykgCQjpYJ-fViV8TyduV-LHG7JjjDsbG3b8NVpLlHoMU_RUx-4N4lFCzrzX-D0DjumqeRiK7C_BTO0cv2LqNIkRsRVOWoMgYjxpJUxaUxIF5ySnElZWvOvA Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:59.501187 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/72e52c52a2524fc4a69040c818109867"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_72e52c52a2524fc4a69040c818109867"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: c25289bd54b24b7dabb300a53ccb4959 Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Volume tempest-invalid-2040715802 could not be found. Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': 'tempest-invalid-2040715802'} Oct 28 11:18:59.503079 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-2040715802 could not be found. (HTTP 404) (Request-ID: req-fe51dcce-9866-42f9-9023-e507bfa5ccc7) Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.504198 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 11:18:59.505751 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-2040715802 could not be found. Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:59.507818 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-2040715802 could not be found. Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-2040715802 could not be found. Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-f81b2f52-4947-4601-8703-5750003a72aa tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Returning 404 to user: Volume tempest-invalid-2040715802 could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-f81b2f52-4947-4601-8703-5750003a72aa tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "DELETE /compute/v2.1/os-volumes/tempest-invalid-2040715802" status: 404 len: 99 microversion: 2.1 time: 0.038073 Oct 28 11:18:59.509404 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 590/1193] 10.222.0.46 () {58 vars in 1218 bytes} [Fri Oct 28 11:18:59 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-2040715802 => generated 99 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 28 11:18:59.512580 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-e50171dd-8b3d-49bc-ab0f-050e8a8dee42 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003767 Oct 28 11:18:59.513075 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 604/1194] 10.222.0.46 () {58 vars in 1140 bytes} [Fri Oct 28 11:18:59 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 28 11:18:59.520854 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-84f405de-c86c-4e58-82bf-acf6e87431af tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003599 Oct 28 11:18:59.521339 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 591/1195] 10.222.0.46 () {58 vars in 1137 bytes} [Fri Oct 28 11:18:59 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Oct 28 11:18:59.529651 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c02b91e4-3245-416d-ab6c-190fa65f7193 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Calling method '.version_select at 0x7f42fa66be50>' {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:59.557249 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.wsgi [None req-c02b91e4-3245-416d-ab6c-190fa65f7193 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] HTTP exception thrown: Volume f1d655f7-6d6a-41f2-9bb7-70432b124666 could not be found. Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c02b91e4-3245-416d-ab6c-190fa65f7193 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Request method failure captured: Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: request: DELETE /compute/v2.1/os-volumes/f1d655f7-6d6a-41f2-9bb7-70432b124666 HTTP/1.1 Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept: application/json Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Accept-Encoding: identity Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Connection: close Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Length: 0 Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Content-Type: application/json Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Host: 10.222.0.46 Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: Openstack-System-Scope: None Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Auth-Token: gAAAAABjW7qjfpujfQAOPHzbbsmU1QeXfqjjL_q19_F1bWkb_jLLPq-iPHwWd2JtykgCQjpYJ-fViV8TyduV-LHG7JjjDsbG3b8NVpLlHoMU_RUx-4N4lFCzrzX-D0DjumqeRiK7C_BTO0cv2LqNIkRsRVOWoMgYjxpJUxaUxIF5ySnElZWvOvA Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Id: None Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Domain-Name: None Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Identity-Status: Confirmed Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Is-Admin-Project: True Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Id: default Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Domain-Name: Default Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Project-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Role: reader,member Oct 28 11:18:59.559193 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Roles: reader,member Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/72e52c52a2524fc4a69040c818109867"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_72e52c52a2524fc4a69040c818109867"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-Tenant-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Id: default Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Domain-Name: Default Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Id: c25289bd54b24b7dabb300a53ccb4959 Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: X-User-Name: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: method: Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: exception: Volume f1d655f7-6d6a-41f2-9bb7-70432b124666 could not be found. Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: action_args: {'id': 'f1d655f7-6d6a-41f2-9bb7-70432b124666'} Oct 28 11:18:59.560911 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: {{(pid=120543) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f1d655f7-6d6a-41f2-9bb7-70432b124666 could not be found. (HTTP 404) (Request-ID: req-001980bf-b23a-4076-a5a6-d2c0f3f479ba) Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.561834 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 28 11:18:59.563186 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f1d655f7-6d6a-41f2-9bb7-70432b124666 could not be found. Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 28 11:18:59.564440 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f1d655f7-6d6a-41f2-9bb7-70432b124666 could not be found. Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f1d655f7-6d6a-41f2-9bb7-70432b124666 could not be found. Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: DEBUG nova.api.openstack.wsgi [None req-c02b91e4-3245-416d-ab6c-190fa65f7193 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Returning 404 to user: Volume f1d655f7-6d6a-41f2-9bb7-70432b124666 could not be found. {{(pid=120543) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: INFO nova.api.openstack.requestlog [None req-c02b91e4-3245-416d-ab6c-190fa65f7193 tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "DELETE /compute/v2.1/os-volumes/f1d655f7-6d6a-41f2-9bb7-70432b124666" status: 404 len: 109 microversion: 2.1 time: 0.034248 Oct 28 11:18:59.565697 dv-cinder-zed-3-iscsi devstack@n-api.service[120543]: [pid: 120543|app: 0|req: 605/1196] 10.222.0.46 () {58 vars in 1248 bytes} [Fri Oct 28 11:18:59 2022] DELETE /compute/v2.1/os-volumes/f1d655f7-6d6a-41f2-9bb7-70432b124666 => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 28 11:18:59.571296 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b16e9f19-ce3e-4eb3-b3cf-342dad5a0b0e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Calling method '.version_select at 0x7f42fa4b8550>' {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 28 11:18:59.597903 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.wsgi [None req-b16e9f19-ce3e-4eb3-b3cf-342dad5a0b0e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] HTTP exception thrown: Volume 8345043d-98ec-4f45-aa1d-17ef3364a11e could not be found. Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b16e9f19-ce3e-4eb3-b3cf-342dad5a0b0e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Request method failure captured: Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: request: GET /compute/v2.1/os-volumes/8345043d-98ec-4f45-aa1d-17ef3364a11e HTTP/1.1 Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept: application/json Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Accept-Encoding: identity Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Connection: close Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Length: 0 Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Content-Type: application/json Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Host: 10.222.0.46 Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: Openstack-System-Scope: None Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: User-Agent: python-urllib3/1.26.12 Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Auth-Token: gAAAAABjW7qjfpujfQAOPHzbbsmU1QeXfqjjL_q19_F1bWkb_jLLPq-iPHwWd2JtykgCQjpYJ-fViV8TyduV-LHG7JjjDsbG3b8NVpLlHoMU_RUx-4N4lFCzrzX-D0DjumqeRiK7C_BTO0cv2LqNIkRsRVOWoMgYjxpJUxaUxIF5ySnElZWvOvA Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Id: None Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Domain-Name: None Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Identity-Status: Confirmed Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Is-Admin-Project: True Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Id: default Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Domain-Name: Default Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Project-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Role: reader,member Oct 28 11:18:59.599849 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Roles: reader,member Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2/72e52c52a2524fc4a69040c818109867"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.46:8080", "publicURL": "http://10.222.0.46:8080/v1/AUTH_72e52c52a2524fc4a69040c818109867"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46/volume/v3/72e52c52a2524fc4a69040c818109867"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.46:9696/networking"}]}] Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Id: 72e52c52a2524fc4a69040c818109867 Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-Tenant-Name: tempest-VolumesNegativeTest-1078660243 Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Id: default Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Domain-Name: Default Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Id: c25289bd54b24b7dabb300a53ccb4959 Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: X-User-Name: tempest-VolumesNegativeTest-1078660243-project Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: method: Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: exception: Volume 8345043d-98ec-4f45-aa1d-17ef3364a11e could not be found. Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: action_args: {'id': '8345043d-98ec-4f45-aa1d-17ef3364a11e'} Oct 28 11:18:59.601612 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: {{(pid=120544) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 8345043d-98ec-4f45-aa1d-17ef3364a11e could not be found. (HTTP 404) (Request-ID: req-4de512ec-7b5d-424a-ad6e-97b5446baf80) Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.602720 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 28 11:18:59.604238 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 8345043d-98ec-4f45-aa1d-17ef3364a11e could not be found. Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 28 11:18:59.605688 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 8345043d-98ec-4f45-aa1d-17ef3364a11e could not be found. Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 8345043d-98ec-4f45-aa1d-17ef3364a11e could not be found. Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: ERROR nova.api.openstack.wsgi Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: DEBUG nova.api.openstack.wsgi [None req-b16e9f19-ce3e-4eb3-b3cf-342dad5a0b0e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] Returning 404 to user: Volume 8345043d-98ec-4f45-aa1d-17ef3364a11e could not be found. {{(pid=120544) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: INFO nova.api.openstack.requestlog [None req-b16e9f19-ce3e-4eb3-b3cf-342dad5a0b0e tempest-VolumesNegativeTest-1078660243 tempest-VolumesNegativeTest-1078660243-project] 10.222.0.46 "GET /compute/v2.1/os-volumes/8345043d-98ec-4f45-aa1d-17ef3364a11e" status: 404 len: 109 microversion: 2.1 time: 0.034335 Oct 28 11:18:59.607200 dv-cinder-zed-3-iscsi devstack@n-api.service[120544]: [pid: 120544|app: 0|req: 592/1197] 10.222.0.46 () {58 vars in 1245 bytes} [Fri Oct 28 11:18:59 2022] GET /compute/v2.1/os-volumes/8345043d-98ec-4f45-aa1d-17ef3364a11e => generated 109 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)